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Executive Summary 



In an effort to evaluate the impact of the SSIs on student achievement and the lessons that 
could be learned from the National Science Foundation’s effort to reform mathematics 
and science education on a statewide basis, research studies identified the technical 
strategies, the political strategies, and the interactions with funders that were critical 
factors in the attempt to effect significant change in student learning over large 
populations. Documents were received on 21 of the 26 SSIs. More intensive data were 
collected via telephone interviews of key personnel in seven of these states and during 
site visits in six other states. Among a number of lessons learned were the following: it 
was vital to incorporate enough flexibility within the design so that information produced 
by research, evaluation, and monitoring could be effectively used — technical lesson; the 
creation of partnerships with policy organizations significantly advanced policy work — 
political lesson; and, SSI leaders and funders needed to develop a shared, in-depth 
understanding of the reform strategies as these fit the local context — interaction with 
funders. 

In addition, an analysis of the NSF’s systemic initiatives compared student mathematics 
test data for grades 4 and 8 in SSI states and non-SSI states with data from State NAEP 
assessments for three testing years, 1992, 1996, and 2000. Comparisons were made of 14 
SSI states and 13 non-SSI states that participated in the State NAEP in each of these three 
testing years. 

The close fit found between improved performance and SSI funding suggests that a 
relationship exists between such initiatives and student achievement. Of equal importance 
is the finding that change is most effective when multiple components are addressed in 
concert: i.e., when the SSIs served as catalysts for other reform efforts that states had 
initiated, they achieved optimum impact. When state policies are aligned with the goals 
of a systemic initiative and when state infrastructure supports teachers and schools as 
they change their practices, reform can result in substantial achievement gains in a 
relatively short time. 
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Summary 



Study of the Impact of Statewide Systemic Initiatives 

Early in the 1990s, the National Science Foundation embarked on an ambitious mathematics and 
science education reform effort that continued throughout the decade. Based on a commitment to 
systemic reform, the Statewide Systemic Initiatives (SSI) Program provided funding to 
qualifying states that enabled them to make simultaneous changes in multiple components to 
achieve improved student learning of challenging content. Over the decade, 25 states and the 
Commonwealth of Puerto Rico received millions of dollars from NSF for up to five years. Eight 
of the jurisdictions received funding for a total of ten years. The present study mined existing 
performance data from the National Assessment of Educational Progress (NAEP) and acquired 
new data from SSI leaders for evidence of the impact of the SSI Program on student learning and 
to determine what could be learned about the strategies, policies, and activities that were needed 
to advance large-scale reform. The study was driven by two main research questions: 1) What 
lessons have been learned about designing, implementing, evaluating, and supporting statewide 
systemic reform? 2) What differences were there on mathematics achievement as measured by 
NAEP between SSI states and non-SSI states over the period 1992 to 2000? The methods and 
findings related to these two questions are presented below. 

Lessons Learned 

In the qualitative analyses, external documents were reviewed on 21 of the 26 SSI jurisdictions. 
Internal documents produced by the SSIs were reviewed for 13 states, with telephone interviews 
conducted of key personnel in seven states, and site visits and more intensive interviews in the 
other six states. Data were analyzed using topical and thematic coding schemes that examined 
the technical and political strategies of the jurisdictions studied. The major conclusions of the 
study and the lessons learned were derived from cross-case analysis. 

The lessons learned are derived from technical strategies and demands, political strategies and 
demands, and interactions with funders. These three areas are not considered independent, but 
identify three important functions that all of the SSIs faced. The SSI leaders learned a great deal 
about statewide systemic reform as a part of the enactment of reform in their states through the 
SSI program. Many of these lessons were similar across SSIs, and both positive and negative 
examples supported the lessons learned. 

Technical strategies. Technical strategies are interventions needed to bring about those changes 
in teaching and learning that result in improved and more equitable student achievement. A 
sound technical strategy is one that: 

• Operationalizes the reform vision through interventions; 

• Monitors and refines the interventions, and provides evidence that they result in 
improved classroom practice and improved student outcomes; 

• Increases capacity within the state to scale up the reform efforts; and 

• Provides evidence that quality and impact are maintained during scale-up. 
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Lessons related to technical strategies and demands were: 

1. Beginning with a manageable scope and scale in design was beneficial. 

2. Establishing data systems to monitor progress, measure impact, and assure quality 
research was at least as important as focusing resources on scaling-up the 
interventions. 

3. Designing interventions that incorporated the flexibility to effectively use what was 
learned through research, evaluation, and monitoring was vital. 

4. Many kinds of capacity need to be developed to initiate and continue reform. 

5. Developing a feasible plan for scaling-up within a reasonable time frame was 
important; scaling-up too quickly can become problematic. 

6. Creating a healthy tension between capacity building for scale-up and the 
achievement of quality control was absolutely necessary. 

Political strategies. Political strategies describe how the SSIs envisioned, and set to work, 
establishing a supportive context for reform. A sound political strategy is one that: 

• Facilitates development of formal policies that provide guidance and incentives for the 
reform vision; 

• Cultivates broad understanding of and support for the reform vision; and, 

• Increases school and district leadership commitment to reform. 

Lessons related to political strategies and demands were: 

1. Housing SSIs within, or forming a partnership with, policy organizations positioned 
many SSIs to engage in policy work. 

2. Involving education policy makers as leaders or partners of an SSI situated the 
initiative to be a natural contributor to policy decisions related to mathematics and 
science education. 

3. Providing an “existence proof’ of high quality, valued service, and contribution in 
one policy area often expanded the role of the SSI in a state’s wider education policy 
arena. 

4. Establishing the understanding and support of mathematics and science leaders in the 
state and education leaders in general increased the likelihood that the SSI would 
become a player by being informed and consulted. 

5. Nurturing relationships with regional and local leaders and stakeholders, including 
superintendents, principals, curriculum supervisors, curriculum committees, and 
parents, was needed to reach schools and classrooms. 

6. Establishing neutral political turf to bring constituencies together benefited the 
initiatives significantly. Maintaining a connection while staying at a reasonable 
distance from existing agencies enabled some initiatives to convene a broad array of 
stakeholders. 

7. SSIs had to balance taking credit and sharing credit for successes with collaborators 
and other reform actors in their state. 
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8. The need and opportunity existed to develop new, expanded leadership for 

mathematics and science education reform in order to expand the reform statewide 
and to sustain the effort into the future. 

Managing interactions with funders. The SSI Program launched a series of NSF programs that 
evolved out of a commitment to provide challenging, meaningful science and mathematics 
education to all students through changes in whole systems of education. It also pioneered new 
relationships between NSF and program awardees in the form of cooperative agreements. The 
critical aspects of how initiative leaders managed their interactions with NSF included: 

• Developing a shared understanding of the strategy for reform; 

• Negotiating appropriate changes to the design; and, 

• Making a case that their initiative was having the desired impact. 

There were three important lessons for managing interactions with funders and sustaining these 
relationships in the future: 

1. The SSI leaders and the funders needed to develop a shared, in-depth understanding 
of the reform strategy as it fit the local context. 

2. Appropriate changes in reform design needed to be negotiated through a shared 
understanding between the initiative leaders and funders, with careful attention to the 
trade-offs and balances associated with these changes. 

3. A shared understanding of the reform strategies, expected impacts over time, and 
long-term outcomes of the initiative was needed to guide the collection, 
interpretation, and reporting of appropriate evidence. 

Findings from the NAEP Analysis 

State NAEP student mathematics achievement data from 1992, 1996, and 2000 and teacher- 
reported information on classroom practices from 1992 and 1996 were analyzed to compare 
student performances and practices in 14 SSI states and 13 non-SSI states. The states included in 
the sample analyzed were all states that had participated in the State NAEP assessments during 
the three years of the study. The states selected, although not randomly chosen, represented a 
cross-section of the states in each group and had characteristics (average public school 
enrollment, per capita expenditure, percent of White students enrolled, and 1992 average 
mathematics achievement) similar to those of all of the states in their respective groups. 

A variety of analytic approaches was used with the State NAEP data to compare and contrast the 
SSI and non-SSI states. Overall achievement, as well as performance for population subgroups, 
was described via means and mean differences between groups. Hierarchical linear modeling 
was used to estimate rates of growth for each group across 1992, 1996, and 2000. Performance 
differences on individual NAEP items were identified using differential item functioning. Scales 
of reform-related instructional practices were constructed from items on the NAEP teacher 
questionnaire, and changes over time were examined with regression methods. Qualitative 
methods were used to identify differences among the SSI states and to relate these features to 
achievement gains. Finally, results of state assessments in three SSI states were compared to the 



IX 




results of State NAEP for those states. The paragraphs below summarize the findings from each 
of these studies. 

Descriptive analyses. The average composite mathematics performance of students at grades 4 
and 8 in the 14 SSI states and the 13 non-SSI states remained nearly comparable over the eight 
years of the study, 1992 to 2000. During this time period, both groups improved on the average 
by 6 scale points at grade 8 and by 6.5 scale points at grade 4. In 1992, grade 8 students in the 
SSI states averaged slightly lower than those in the non-SSI states (1.2 scale points difference). 
All other differences between the two groups of states by year and grade were .7 scale points or 
less. Thus, there were no differences between the SSI states and the non-SSI states in average 
mathematics composite scores for each of the three testing times, except for grade 8 in 1992. 

In 1992, there was considerable variance in mean performance among the states within the SSI 
group and within the non-SSI group: both groups included high-performing states and low- 
performing states. Over the eight years, the variance in mathematics performance among states 
decreased. Empirical Bayes and Bayesian Analyses confirmed that the average mathematics 
performance by SSI states at both grades 4 and 8 began below that of the non-SSI states in 1992, 
but increased at a faster annual growth rate, although not statistically significant, than the non- 
SSI states. 

Population subgroups. There were no differences by gender between SSI states and non-SSI 
states in mathematics performance over the eight years from 1992 to 2000. 

There is evidence that Black students in SSI states made relatively higher gains than those in 
non-SSI states between 1992 and 1996. Hispanic students in SSI states made relatively higher 
gains than those in non-SSI states between 1996 and 2000. This evidence is apparent in both 
cross-sectional data and growth in performance from grade 4 to grade 8 by the same cohort of 
students. Black students in SSI states made a slightly higher gain from grade 4 in 1992 to grade 8 
in 1996 than did White students on the Number/Operations strand and the Algebra/Functions 
strand. Over those four years, the performance of White students and Black students from SSI 
states was more similar than for these two populations in non-SSI states. However, over the next 
four years this trend was reversed, with the gap between White students and Black students in 
2000 being smaller for non-SSI states than for SSI states. For Hispanic students, the finding was 
reversed. The gap between growth in performance from grade 4 to grade 8 of White students and 
Hispanic students was less for SSI states than for non-SSI states between 1996 and 2000 and was 
greater between 1992 and 1996. 

Differential item functioning (DIF). Some differences in the underlying mathematical 
constructs of the performance of students from the SSI states compared to those from the non- 
SSI states were detected. At both grades 4 and 8, when the performances of students with equal 
abilities were compared, students from SSI states performed higher on items from the Data 
Analysis content strand and items requiring problem solving. Both item types represent areas that 
have been emphasized in reform mathematics over the 1990s. Students from SSI states also 
performed better on an increasing number of multiple-choice items from 1992 to 2000. This 
finding, along with a reduction in the number of DIF items in more widely covered content 
strands of Number/Operations and Algebra/Function, indicates that students from SSI states 
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improved in performance in relation to students from non-SSI states who were of equal ability 
both on a greater number of reform topics and on more traditional measures. 

Scales of reform-related instructional practices. Classroom practices in SSI states incorporated 
a greater number of reform practices than classroom practices in non-SSI states. We analyzed six 
reform indicators — three on classroom practices and three on teachers’ knowledge and 
professional development. As expected, SES (socioeconomic status) was the primary predictor 
of a states’ mean mathematics composite in both 1992 and 1996. SSI states averaged 
significantly higher on an indicator of the relative emphasis on reasoning and communication at 
both grades 4 and 8 in 1996. In a regression model, this indicator was predictive of the mean 
State NAEP mathematics scores. Teachers in SSI states compared to those in non-SSI states 
reported giving students more opportunities for mathematical discourse in both 1992 and 1996. 
However, the difference was not significant. The use of criterion-referenced tests (CRT) in at 
least two grades from grades 3 through 8 was used as another variable for describing reform 
within states. Both SSI status and CRT were related to achievement gains across the three State 
NAEP administrations (1992, 1996, and 2000). The gains were the largest among states with 
criterion-referenced tests. 

Use of qualitative research to analyze NAEP performance. We employed qualitative 
methodology to understand more fully what might explain the differences in performance by the 
14 SSI states in the longitudinal trend sample. In addition to the mathematics performance data 
from the State NAEP, we used data from a number of sources. To gather more information on 
the independent variable for the time covered by the assessment data, we interviewed state 
mathematics supervisors and SSI leaders; we also reviewed documents to provide information on 
the percentage of teachers in the state reached by reform in 1990, 1992, and 1996, emphasis 
given to components of reform, and relative emphasis given by the SSI on the five State NAEP 
content strands. Information from these SSI state reports was supplemented with data from a 
policy analysis of state SSIs, evaluations of the SSI program by other analysts, and annual 
surveys of state student assessment programs. We also consulted the Horizon Research team on 
their findings from the SSI states in the trend sample. Using these data as a basis for our analysis, 
we divided the SSI states into three groups based on State NAEP mathematics performance for 
the three testing times in both grades 4 and 8 — Steady Increase, Some Increase, and Little/No 
Increase. 

Overall, our findings are consistent with the underlying theory of systemic reform. State 
assessments and accountability policies appear to be strong factors in improved student 
performance. Furthermore, we found that state policies aligned with the goals of a systemic 
initiative, along with a sufficiently strong statewide infrastructure to support teachers and schools 
as they change their practices, can result in substantial achievement gains in a relatively short 
time. More specifically, we found: 

• Statewide achievement gains across four years were more likely to be evident when 
reform efforts addressed state policy as much as or more than teachers and classroom 
practices. 
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• Statewide assessment policies and practices seemed to be important components of 
systemic reform. The existence of a state assessment program seems to be related to 
statewide achievement gains, particularly when criterion-referenced tests were used. 

• There is some indication that when state policies were not supportive of SSI goals, 
reform efforts were compromised or even undermined. 

• When assessments were aligned with the goals of the SSI, reform-related instructional 
practices increased; when they were not aligned, reform-related instructional practices 
did not change, or decreased. 

• States with a strong infrastructure prior to the SSI generally had steady gains in 
achievement. States with large increases in reform indicators during the SSI were able to 
steadily increase achievement with Phase II funding. Achievement gains from 1996 to 
2000 were unlikely to occur in SSI states that did not receive Phase II funding. 

• In all SSI states, the alignment of state frameworks and assessment with the SSI goals 
appeared to be an important influence on statewide student achievement. 

State assessments and State NAEP. The framework for State NAEP assessments in 1990 
through 2000 was designed to use a number of sources that included state and district standards 
and the National Council of Teachers of Mathematics Curriculum and Evaluation Standards for 
School Mathematics . Although the State NAEP provides information on a range of mathematics 
performance, it was not designed for precise measurement of curriculum standards and 
frameworks from any one state or reform in mathematics in any one state. Students also do not 
have the same motivation to perform on the State NAEP as they do on state assessments where 
the results have some meaning to them. The assessments designed and administered by the state 
should be in a better position to do this. We conducted a focus study comparing results from 
assessments administered by three of the SSI states — Texas, Maine, and Massachusetts — using 
the State NAEP results to verify the findings attained from the State NAEP and as a basis for 
closer inspection of the relationship between an SSI intervention among schools within a state 
and mathematics performance. 

In trying to use data from state assessments, we were confronted with a number of issues that 
included change in the state assessments over the time period, lack of year-to-year data, and 
insufficient documentation of data needed for longitudinal analyses. Our findings were mixed. 
For the time period between 1992 and 1996, the State NAEP results and the state assessment 
results for Texas and Massachusetts were comparable. However, the Maine state assessment for 
grade 4 showed improvements that were not apparent on the State NAEP. For the 1996 to 2000 
time period, only state assessment data from Texas could be used in our analysis. During this 
period, both State NAEP and the state assessments indicated some improved performance, but 
the Texas state assessment indicated substantially more improvement than did the State NAEP, 
similar to the previous results for Maine. Even though the state assessment scores in 
Massachusetts from 1992 to 1996 showed little gain, the cohort of schools with the most intense 
SSI involvement over this period did show improved scores. Thus, the State NAEP can be 
sensitive to some large group changes in performance as verified by state assessments, but is less 
sensitive to more subtle effects when reform efforts target specific subpopulations. 

While state assessment data proved to have potential for the study of reform efforts within states, 
it was determined that continuity of content and program design is essential for such studies. 
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Furthermore, test designs that reflect the knowledge, skill and cognitive development of 
disciplines as well as psychometric scales that allow for adequate measurement of growth are 
required if these state assessments are to detect achievement improvements over time. 

Conclusions 

We did not design this research to be, nor did we have the resources to conduct, the definitive 
study on the impact of the Statewide Systemic Initiatives. However, the study supports the 
finding that a tremendous amount of learning about how to engage in large-scale reform took 
place over the duration of the SSI program and that in states having an SSI, we found an 
increased rate of learning by students. The findings in this study have produced a number of 
lessons learned that are directly applicable to any attempt to make significant changes in student 
learning over a large population. We learned that it is not only critical to consider the technical 
issues concerning the functioning of a program, but it is essential to address the political 
decisions within the state and negotiations with the funder in order to garner the support 
necessary to sustain an effort long enough for a measurable impact on student learning to be 
achieved. 

It was impossible in this study to isolate the specific impact of an SSI on student learning. When 
SSI states were studied as a group and compared to non-SSI states, there was evidence that 
student scores from 1992 to 1996 to 2000 in SSI states increased at a faster rate than did student 
scores in the non-SSI states. The variation among SSI states was as great as among non-SSI 
states. It was clear that SSI states with Phase II funding accelerated the rate of learning over the 
time period from 1996 to 2000, whereas the SSI states that did not receive continued funding and 
the non-SSI states as a group maintained or declined in the rate of learning over this time period. 
The close fit between improved performance and SSI funding suggests a possible relationship 
between the statewide systemic initiatives and student performance, but it was impossible to 
discount other alternative hypothesis including a selection bias. It was also clear that teachers in 
SSI states were using a greater number of reform practices than those in non-SSI states and that 
students from SSI states were performing more favorably on those mathematics areas that were 
given greater emphasis in reform mathematics curricula — Data Analysis and Problem Solving. 
The findings from this study are very compatible with the theory of systemic reform and the need 
to change multiple components in concert rather than independently. The findings are consistent 
with NSF’s vision that the SSIs serve as catalysts for other reform efforts in states. Those states 
with a more developed infrastructure prior to the SSI were able to take greater advantage of the 
SSI funding. Overall, the SSI program was related to an increased rate of student performance in 
some states. The variation in improved student performance among SSI states appeared to be 
related to prior conditions in their education systems, accountability, and duration of funding. 

A number of methodologies were used to complete this study. The SSI leaders interviewed were 
a deep source of information on implementing large-scale reform. The State NAEP proved to be 
a viable source of data that could be used to study differences among states and compare SSI 
states with non-SSI states. Even though only about 60% of the states in each of these groups 
participated in the State NAEP for 1992, 1996, and 2000, the states that did participate were 
representative of the larger groups. An important condition for this study was our capacity to 
secure data and information on the nature and the quality of the SSI implementation. For this 
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information, we drew heavily upon the work of researchers who received funding from NSF to 
describe and analyze the implementation of the SSI program — SRI, the National Institute for 
Science Education, RAND, the Council of Chief State School Officers, the Consortium for 
Policy Research in Education, COSMOS, and Abt Associates. The study and its findings were 
greatly enhanced by combining both qualitative and quantitative methodology. 




Part I: Background of the Study 



The SSI Program 

The Statewide Systemic Initiative (SSI) Program began in 1990 with the release of a solicitation 
by the National Science Foundation’s Directorate for Science and Engineering Education (NSF, 
1990). NSF sought proposals “for projects intended to broaden the impact, accelerate the pace, 
and increase the effectiveness of improvements in science, mathematics, and engineering 
education in both K-12 and post-secondary levels.” 

Noting that efforts to work on individual parts of the system were unlikely to lead to the desired 
breadth and depth of reform, NSF called for proposals that coordinated all of the necessary 
components of systemic change, including: 

• Organizational structure and decision-making; 

• Provision and allocation of resources; 

• Recruitment and preparation of teachers and college faculty; 

• Retention and continuing professional development of teachers and other professional 
personnel; 

• Curriculum content and learning goals; 

• Articulation within the system; 

• Delivery of instruction, including the use of educational technology; 

• Facilities and equipment; 

• Assessment of student achievement; and 

• Accountability systems. 

NSF’s intent was to “stimulate and catalyze” high quality reform efforts, supporting individual 
states with $1-2 million per year for five years. Proposers were encouraged to involve a broad 
array of stakeholders, including scientists and mathematicians, and business and community 
representatives, as well as local school system decision makers and leaders of parent and other 
community-based organizations. 

The SSI Program signaled a landmark commitment to excellence and equity in mathematics and 
science education for all students. The solicitation sought to fund projects that would “increase 
the knowledge of science and mathematics acquired by all students at all education levels.” 

Ten SSI awards were made in the first cohort in 1991 (Connecticut, Delaware, Florida, 

Fouisiana, Montana, Nebraska, North Carolina, Ohio, Rhode Island, and South Dakota); 
followed by 11 in the second cohort in 1992 (California, Georgia, Kentucky, Maine, 
Massachusetts, Michigan, New Mexico, Puerto Rico, Texas, Vermont, and Virginia); and 5 in the 
third cohort in 1993 (Arkansas, Colorado, New Jersey, New York, and South Carolina). 

Initially, states were allowed to limit their plans to either mathematics or science education, in a 
subset of the grades K-16; as the program evolved NSF required SSIs to address both 
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mathematics and science education in all of the grades, and negotiated with the original awardees 
that had more limited plans to expand the scope of their work. In addition, beginning in 1995, 
NSF specified a set of key “drivers” of systemic reform, asking each SSI to report its progress in 
terms of: (1) implementation of comprehensive, standards-based curricula; (2) development of a 
coherent set of policies to support high quality science and mathematics education; (3) 
convergence of the use of resources in support of science and mathematics education; (4) broad- 
based support for the reforms; (5) evidence that the program is enhancing student achievement; 
and (6) evidence that the program is improving the achievement of all students, including those 
that have historically been underserved. 

NSF set up a support structure for the SSI program, funding the Education Development Center 
to provide technical assistance to the SSIs; SRI International to do a program evaluation, 
including a number of case studies; and Abt Associates Inc. to assist NSF in monitoring the 
quality of the initiatives. In addition, NSF required each SSI to submit annual progress reports, 
and to participate in a “reverse site visit” midway through their funding period to “make their 
case” for continuation of project funding to NSF and a panel of reviewers. 

Given the large expenditures involved, NSF had a stake in monitoring their reform efforts quite 
closely, and, in fact, SSIs that were deemed to be making insufficient progress towards systemic 
reform were discontinued during their initial five years. These SSIs included Florida, North 
Carolina, Rhode Island, and Virginia. 

As each cohort approached the end of its funding period, NSF invited the projects to apply for 
additional funds (up to $1.4 million per year for up to five years) to enable them to scale-up their 
efforts statewide. A total of eight SSIs received these Phase II awards, including Connecticut and 
Louisiana from the first cohort; Massachusetts, Texas, Vermont, and Puerto Rico from the 
second cohort; and New Jersey and South Carolina from the third. 



Previous Studies of the Statewide Systemic Initiatives and Systemic Reform 

Since 1991, the National Science Foundation has funded a number of research and evaluation 
projects to study the Statewide Systemic Initiatives. In addition, policy and education 
researchers have tapped the SSI Program, among others, in order to investigate instantiations of 
systemic reform. The approaches and findings of past and ongoing research on the SSIs and 
systemic reform more generally provided a strong foundation upon which the current study was 
built. 

A great deal of descriptive, interpretive, and analytic work on the SSIs was conducted through 
the national SSI evaluation contract awarded to SRI International and through the contract for 
monitoring awarded to Abt Associates, Inc. Additionally, as a part of the same competition 
under which our study was funded, NSF awarded a number of research and evaluation grants to 
study the various programs of Education Systemic Reform, including the SSI Program, or cross- 
cutting aspects of systemic reform across programs. 
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Leadership 

The evaluation work of SRI International (summarized in Zucker, Shields, Adelman, Corcoran, 

& Goertz, 1998) highlighted the importance of leaders in defining what the SSI in each state 
would entail, because the program solicitation invited a wide variety of designs. SRI’s 
evaluation and the Wingspread Conference of SSI leaders conducted during Phase I of the SSI 
Program (Horizon Research, Inc., Inverness Research Associates, & Westat, Inc., 1994) both 
noted the importance of collaborative and shared leadership in the initiatives, because different 
kinds of leadership were required for the many demands of systemic reform at the state level. 
Looking back on nearly a decade of the SSI Program, the Council of Chief State School Officers 
(CCSSO, 2000) reported that leadership needs of systemic reform changed as the initiatives grew 
and matured. One possible consequence was noted in SRI’s findings about the program; namely 
that the SSIs became strong training ground for new leaders in the states, leaving a legacy of 
leadership capacity in mathematics and science education. 

Strategies 

SRI (Zucker et al., 1998) identified eight basic strategies of the SSIs that fell into two broad 
categories: (1) focus close to the classroom on teachers and schools, and (2) focus close to the 
state system and infrastructure on districts, regions, and the state. Two important findings from 
the SRI evaluation were that individual SSIs used multiple strategies with different emphases, 
and that SSIs could be built either with one of the two primary foci, or they could incorporate a 
balance of both. The study reported that no strategy or combination of strategies could be 
determined to be the “one right way,” but rather the approaches reflected attempts to fit the SSIs 
to the particular state contexts in which the initiatives operated. In a later study of the Phase II 
SSIs, COSMOS Corporation (2002) found that the originally conceived sequence of systemic 
education reform — standards, framework, curricula, assessments — was not followed in all SSIs, 
nor was it the only possible successful model for systemic reform. Beginning with a policy 
instrument, such as assessment, or beginning with a more bottom-up approach might be viable 
alternatives. Similarly, Ware, Richardson, and Kim’s (2000) study of the Urban Systemic 
Initiative/Program (USI/P) highlighted professional development, curriculum, and student 
outcomes as key factors in systemic reform in urban settings, but they did not find that the 
USI/Ps as a group treated these factors in the same sequence or in the same way in their systemic 
reform programs. 

Models of Systemic Reform 

Researchers and evaluators have constructed models of systemic reform for describing and 
analyzing SSIs. The SRI evaluation (Zucker et al., 1998) built a model of the education system 
that included multiple components that SSI activities might influence, as well as outcomes that 
SSI activities might impact. The model is an especially useful tool for considering where the 
combination of strategies employed by an SSI was targeted and where it was most likely to have 
discernible effects. Clune’s (1998) study offered a model using a “continuous causal sequence” 
linking systemic reform activity to systemic policy to the implemented curriculum for all 
students to student achievement. The study presented the model as a way to test the theory of 
systemic reform, and concluded that the SSI states that were studied supported the model with 
both positive and negative examples. The COSMOS Corporation study (2002) provided another 
model of systemic reform in action. The COSMOS model identified a number of critical 
components of systemic reform, and included measures of alignment of each component with the 
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goals of systemic reform, of scale-up of reform to schools and/or classrooms, and of change in 
student performance. The model was developed to analyze systemic reform at the state level, 
over time, as a non-linear process. Within this model the activities of the SSI were investigated 
not as inputs to the system, but as one of many possible agents acting on one or multiple 
components of the system. 

Technical and Political Demands 

A number of past studies have highlighted the dual technical and political demands of systemic 
reform. The proceedings of the Wingspread Conference (Horizon Research, Inc. et al, 1994) 
identified technical and political aspects of reform that emerged within the SSIs themselves as 
they sought to define and operationalize their reform visions in plans for systemic reform. The 
proceedings also described additional technical and political demands the SSIs encountered as 
they implemented their plans in the states, districts, schools, and classrooms. SRI’s evaluation 
(Zucker, Shields, Adelman, & Powell, 1995; Zucker et al., 1998) also noted the need for attention 
to both the technical and political aspects of reform in order for SSIs to achieve their ambitious 
goals. Technical demands manifested themselves, for example, in terms of designing 
professional development consistent with high standards for curriculum and teaching (Zucker et 
al., 1998; CCSSO, 2000; Ware, Richardson, & Kim, 2000), identification or development of high 
quality instructional materials (Zucker et al., 1998), and maintaining quality control during the 
scaling-up process (Zucker et al., 1998; Corcoran, 1997; Massell, Kirst, Hoppe, 1997). Political 
demands manifested themselves, for example, in terms of establishing coherence and continuity 
with past and current projects within states (Clune, 1998; CCSSO, 2000), forging partnerships 
with key agencies and power brokers within the state (Zucker et al., 1998, CCSSO, 2000; Clune, 
Millar, Raizen, Webb, Britton, Bowcock, Gunter, & Mesquita, 1997a, 1997b), and initiating or 
contributing to policy alignment efforts in the state (Zucker et al., 1998; CCSSO, 2000; Goertz, 
Floden, & O’Day, 1995a, 1995b). 

Equity 

Equity is a key factor in systemic reform and remains at the heart of the greatest challenges in 
education. Massed and colleagues (1997), the Wingspread Conference (Horizon Research, Inc. 
et al, 1994), SRI (Zucker et al., 1998), Ware and colleagues (2000), and the National Institute for 
Science Education (NISE) (Clune et al., 1997a, 1997b; Webb, Century, Davila, Heck, & Osthoff, 
in preparation) ad noted that great attention has been given to equity in systemic reform efforts, 
most obviously in the form of defining and fostering commitment to high standards of 
achievement for ad students. However, each of these studies also noted that high standards for 
ad and even many strong components of systemic reform efforts around high standards for ad do 
not necessarily result in equity of opportunities and outcomes. These equity goals can get lost in 
rhetoric, and it is up to those leading systemic reform to assure that they do not. 

Evaluation 

Evaluation has been of keen interest in research on systemic reform. NISE (Clune et al., 1997a, 
1997b; Webb et al., in preparation), especially, conducted work to document and analyze the role 
of evaluation in the SSIs and other systemic reform programs. SRI (Zucker et al., 1998) reported 
critical roles of local evaluators in the SSIs, including quality control, tracking progress, and 
demonstrating impacts on classroom instruction and student learning. The Wingspread 
Conference (Horizon Research, Inc. et al, 1994), CCSSO (2000), NISE (Clune et al., 1997a, 
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1997b; Webb et al., in preparation), and the Consortium for Policy Research in Education 
(Goertz et al., 1995a, 1995b; Massell et al., 1997) surfaced concerns that traditional evaluation 
roles and models were not well suited to systemic reform, and that more dynamic, flexible, 
ongoing, and interactive roles were required in order to promote greater reflection and informed 
decision-making at all levels of an education system engaged in systemic reform. 

Student Outcomes 

The “Mosaic” study conducted by RAND (Klein, Hamilton, McCaffrey, Stecher, Robyn, & 
Burroughs, 2000) has provided some evidence that the use of the reform-based instructional 
practices generally advocated by the SSIs has a fairly consistent, albeit weak, positive 
relationship with student achievement. Two important caveats for evaluation around student 
outcomes have been raised in previous studies. First, state assessment systems must be carefully 
understood in terms of their alignment with systemic reform goals and interventions before 
including the results of these assessments in evaluations (Laguarda, Breckenridge, Hightower, & 
Adelman, 1994; Klein et al., 2000). Second, it may be the case that impacts on student outcomes 
will show up only after systemic reform has had a substantial amount of time to infuse the system 
(COSMOS Corporation, 2002; Webb et al., in preparation). Also, impacts on student 
achievement, in particular, may appear either more rapidly, or may be more easily attributed to 
the actions of the SSI when strategies focused close to the classroom are undertaken, as opposed 
to strategies focused close to the state system (Zucker et al., 1998; CCSSO, 2000). These results 
do not, however, necessarily imply that the close-to-the-classroom focus should be categorically 
preferred. Other strategies may result in greater student achievement gains in the long-term. 

Scaling-Up and Sustaining Systemic Reform 

Finally, as the SSIs and systemic reform as a movement have matured, researchers have turned 
attention to how the initiatives have attended to scaling-up and sustaining their interventions and 
outcomes. Many cases of activities and pieces of systemic reform spreading and being sustained 
have been evident, but not entire, coherent initiatives (Corcoran, 1997). Capacity building for 
long-term and broad-scale change, as well as attention to reforming the system infrastructure for 
capacity building among teachers, administrators, and service providers are needed to scale-up 
and sustain systemic reform (Goertz et al., 1995a, 1995b; Massell et al., 1997; Corcoran, 1997; 
Century & Fevy, 2002). Consistency in the policy domain and among education leadership are 
required both to guide the long-term nature of systemic reform and to build and maintain the 
political, professional, and public support, including finances, required to conduct systemic 
reform (CCSSO, 2000; COSMOS Corporation, 2002; Century & Fevy, 2002). 

The Impact of the Statewide Systemic Initiatives Program: Fessons Fearned contributes to this 
body of research and evaluation literature. Our focus on the strategic planning, decision-making, 
and thinking behind the SSIs is intended to be distinct from the strategies of the SSIs. We see the 
strategies as a way of describing what the initiatives did and what they intended to have happen 
as a result. In contrast, we see the strategic planning, decision-making, and thinking of the 
leaders as the reason and rationale behind choices, trade-offs, sequencing, communicating, and 
evaluating. Strategies and strategic planning, decision-making, and thinking are intertwined in 
any initiative. SRI’s efforts to document and understand strategies provided a key basis for our 
work. Our understandings of what the SSIs did, how they did it, and to what effect were strongly 
informed by the work of other researchers and evaluators. Additionally, we benefited from the 
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challenges to systemic reform and hypotheses generated in other work to guide our study and 
inform our interpretations. The contribution of our work is to describe, interpret, and analyze 
why the SSI leaders chose to do what they did and what they have learned about the 
consequences of their decisions in retrospect. Along with the work of our Wisconsin Center for 
Education Research (WCER) partners in the study, this research combines an inside-out, close-up 
view of the thinking and reflections of leaders of the SSIs with an outside-in, large-scale view of 
effects of the SSI on instructional practice and student achievement. 



Conceptual Background of This Study 

Education reform that targets isolated components of the education system, even when 
successful, often does not endure more than a few years due to system pressures supporting a 
return to the status quo (Berman & McLaughlin, 1978; Fullan, 2001; Smith & O'Day, 1991). 
Even more sophisticated efforts toward education reform, when applied in education systems, are 
often tied to the vision and leadership of one individual or to one source of funding. With the 
departure of a critical individual or the loss of a particular source of funding, reform within an 
education system rarely endures (Wiles, 1993). 

NSF’s Statewide Systemic Initiatives Program intended to address these two critical 
shortcomings of past education change efforts through systemic reform. First, the premise of 
systemic education reform is that isolated efforts focused on components of the system are 
insufficient for deep and lasting change. Rather, it is necessary to have a clear and shared vision 
and to target multiple, related aspects of the education system simultaneously and in a 
coordinated fashion in order to create deep and lasting change in the process and outcomes of 
mathematics and science education for all students. Second, although leadership and funding (or 
other resources) play major roles, systemic reform aims to build capacities in the education 
system that will renew leadership, funding, and other vital driving forces for educational change. 
Ensuring support for an ongoing process of improvement is a vital part of systemic reform. 

The Study of the Impact of the Statewide Systemic Initiatives Program: Lessons Learned focused 
on the design, implementation, and evaluation of the SSIs, specifically on the thinking behind 
how the SSIs were conceived and carried out. We grounded the study in an understanding that 
the initiatives funded through the SSI Program were intended to create substantial and enduring 
reform in education policy, administration and management, support services, and teaching and 
learning. Conceptually, we looked to literature on systems change and sustainable development 
in an array of fields: education, business, non-profit leadership, agriculture, ecology, and 
national development. Despite differences in the traditions of research and practice in these 
fields, many clear parallels were evident in the fundamental principles — infrastructure 
development, capacity building, and equity — underlying how systems change is conceived. 
Additionally, the literature in these fields offers perspectives that we found helpful in considering 
what aspects to investigate and how to interpret our data about the thinking behind the SSIs. 

The SSIs represent a variety of approaches to reforming state education systems in mathematics 
and science, and in some cases technology, which reflected differences in the underlying 
planning, decision-making, and thinking of the leaders about what would constitute systemic 
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reform. Also, successes in changing targeted aspects of state education systems or in moving 
whole systems toward a reform vision varied among the SSIs. Despite these variations, we did 
not attempt to test the hypothesis that more strategic SSIs lead to greater success in advancing 
systemic reform. In fact, this hypothesis runs up against circular logic: 

Any organization that succeeds must have an effective strategy, because a defining 
characteristic of organization success is to follow a game plan that produces results. In 
contrast, a failing organization by definition has an ineffective strategy or else it would 
not be failing. Unfortunately, what makes for an effective strategy is never very clear. 
There are many maxims and cliches in strategic management, but few hard-and-fast rules 
to shape a course of action. (Goldsmith, 1996) 

Our study, then, was one of exploration and hypothesis building. We dedicated ourselves to 
investigating and describing how leaders of the SSIs envisioned an initiative that would 
profoundly transform mathematics and science education, but would fit within their state’s 
context and history of education. We also sought to understand the rationale behind how the 
leaders designed and carried out the SSIs, and to illuminate what worked well, and what did not, 
and why. 

Conceptions of what it means to be strategic in other fields aided us in honing our thinking about 
what it would mean for an SSI to be strategic. For example, in an article on international 
development, Goldsmith (1996) proposed a basic set of criteria for understanding whether an 
organization is strategic in its planning, decision-making, and thinking. In order to be considered 
strategic, according to Goldsmith, an organization has to attend to three fundamental points. 

First, the organization has to know what it has the capacity to do well, and know where it either 
has or can create access to apply its capacities. Put simply, a strategic organization knows what 
it can supply and where there is demand, or where demand can be created. Second, the leaders of 
the organization scrutinize all aspects of the environment — technological, economic, political, 
and social — that affect the organization's ability to pursue its goals. A strategic organization 
needs leaders who come to know intimately the context in which the organization operates. 

Third, the organization has to establish and maintain a fit between important contextual factors 
and internal decisions. 

A finer distinction made in the literature helped us to refine our conceptual basis further. Three 
notions of how an organization and its leaders act strategically have been highlighted and 
distinguished: strategic planning, strategic decision-making, and strategic thinking. 

Strategic planning is conceived of as a blueprint for the actions an organization proposes to take 
in the future to fulfill its mission (Fogg, 1999). Strategic plans are roadmaps for action that 
describe the steps, both big and small, that an organization proposes to undertake in pursuit of its 
goals. 

Strategic decision-making focuses on the processes by which decisions are made within 
organizations, both in establishing and in adjusting strategic plans (Hickson, Butler, Cray, 

Mallory & Wilson, 1989). Investigating strategic decision-making entails inquiring into the 
internal processes by which organizations carry out their work. Researching these processes 
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especially focuses on how the organization and its leaders: (1) understand the progress of 
implementation of its plans; (2) set benchmarks, measure change, and analyze impacts; and (3) 
modify plans in light of information about implementation and impacts. 

Finally, strategic thinking is defined as a broader concept that encompasses strategic planning 
and decision-making, and captures other aspects of organizational leadership and functioning. 
According to Koteen (1989) strategic thinking “attempts to provide ... direction, guide priority 
use of resources, set standards of excellence, cope with environmental uncertainty and change, 
and provide an objective basis for control and evaluation.” Shrivastava (1985) and Goldsmith 
(1996) argue that leaders who practice strategic thinking view problems holistically, rather than 
categorically, and seek solutions expansively, proactively, and opportunistically, rather than 
resorting to selection among predefined options. 

The more inclusive concept of strategic thinking was appealing for examining the SSIs, both 
because it could include strategic planning and decision-making, and because it would support 
investigation of two central features of systemic reform. Earlier studies of the SSIs highlighted a 
critical duality in how leaders thought about the initiatives. In proceedings from a conference of 
SSI leaders fairly early in the life of the program, this duality was identified, “Systemic reform is 
as much a political enterprise as it is a technical one” (Horizon Research, Inc. et al, 1994, p. vi). 
Similarly, in the evaluation of the SSI program, Zucker and colleagues (1995, p. 47) noted that 
“the difficulty of systemic reform stems from the fact that it presents both technical and political 
challenges. Technically, it requires tackling the toughest problems in a complex system of 
education. . . . Politically, systemic reform requires gamering professional and public support for 
the change agenda.” In their final report, the SRI evaluation team (Zucker et al., 1998, pp. xi-xii) 
identified “many technical and political challenges” in the work of the SSI, noting that 
“outcomes were the results of the strategic choices of the SSIs: they focused on what was 
familiar, what they understood would be effective, what would have a payoff in the medium 
term, and what would be politically acceptable.” Investigating the strategic thinking underlying 
the SSIs appeared to provide the best conceptual basis for collecting data, analyzing information, 
and interpreting results. Focusing on strategic thinking facilitated research on both the technical 
work of designing, carrying out, and tracking the implementation and impact of interventions 
(some of which is captured within the narrower concepts of strategic planning and strategic 
decision-making); and the political work of forging connections, anticipating barriers and 
opportunities, and creating openings for the initiative to move forward towards its goals. 

We specifically investigated how the SSI leaders thought about four areas of their work, both in 
terms of how they understood the existing state education system in mathematics and science, 
and in terms of how they determined what the SSI would do. These four areas were: (1) 
building teacher capacity; (2) building infrastructure for support services; (3) aligning policy with 
the reform vision; and (4) gaining support/avoiding opposition for the reform. We considered it 
not so important what an SSI chose to focus on within or across these areas, but rather whether or 
not it focused on areas that were complementary to the needs of its particular state and the 
context within which it was operating. In our view, an SSI’s approach was strategic to the extent 
that the leaders were able to analyze the needs of the state’s context and match the initiative’s 
expertise, efforts, and emphases to the needs of the environment such that all four key areas were 
being addressed and advanced in a coordinated fashion within the state, and were aligned with 
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the reform vision the SSI espoused. It is this notion of strategic that we sought to understand and 
describe in the study. 

Within our framework of strategic thinking, we examined the literature further to determine 
specific factors about which we should be collecting information and against which to interpret 
our findings. Again, examining the idea of strategic thinking across a broad array of fields was 
helpful. Although the language used to describe these factors differed considerably across fields, 
certain common ideas and interpretive lenses emerged across fields that appeared especially 
applicable to the nature of the SSIs. We distilled our list of important factors to consider in data 
collection, analysis, and interpretation down to six: 

1. Backgrounds of key players and organizations; 

2. Vision; 

3. Critical targets of change; 

4. Trade-offs and balance; 

5. Flexibility to deal with uncertainty; and 

6. Internal and external politics. 

Backgrounds of Key Players and Organizational Structures 

Naturally, the expertise and past experiences of individuals and organizations shape how they 
think about the present and possible futures. Pfeffer & Salancik (1978) suggest that people's 
worldviews have a tremendous impact on the kinds of things their organizations view as 
important and that these worldviews are shaped by past experiences. Further, they argue that 
organizational structures reflect the past professional experiences of their leadership. 
Accordingly, the backgrounds and environments of key individuals and organizations leading the 
SSIs were likely to have a filtering, if not directing, influence on strategic thinking about what 
areas to address and how to address them as the initiatives were planned and carried out. Also, 
the nature of the SSI organization itself, both in terms of who its leaders were and how it was 
situated within the state, would be of interest to identify the benefits and challenges of the many 
structures the SSIs employed to house and operate the initiatives. 

Vision 

Virtually all the literature we examined, across fields, emphasized the importance of vision in 
leadership. Bennis and Nanus (1985) captured the importance of vision particularly well, noting 
that effective leaders use vision as an instrument to move organizations and people toward future 
conditions. Leaders engaged in strategic thinking about systems change, applying their view, 
create the promise of potential opportunities and empower people and organizations to pursue 
those opportunities. Having and communicating a distinct vision can be a powerful enabling 
mechanism to motivate people and organizations to act. We pursued an understanding of how 
leaders of the SSI conceived, shared, built support for, and utilized a reform vision to craft the 
initiative and move the reform forward over time. 

Critical Targets of Change 

Strategic thinking about changing a system, although expansive in its overall view, must become 
more reductionist as reform efforts are crafted. This can be accomplished by targeting specific 
aspects of the system in need of change, specifying how they are to be changed, and envisioning 
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how these specific changes contribute to the larger picture of changing the whole system. 
Understanding what strategic thinkers view as the critical targets of change, the means by which 
they intend to change certain aspects of the system, the nature and extent of the changes they 
pursue, and the connections among discrete changes that they believe will reform the system will 
highlight what they see as the necessary and sufficient resources, processes, structures, and 
functions required to reform the system. Especially as strategic thinkers consider what it takes to 
scale-up and sustain reforms, the idea of critical factors to change — in a particular direction and 
to a particular level — become especially important (Dovers, 1990; Farrell & Hart, 1998; 
Goodland & Daly, 1996; MacDonald, 1996; Orians, 1990). We hoped to understand in the SSIs 
what parts of the system leaders saw as “pressure points,” “leverage points,” and “high yield 
areas” where changes would have lasting and far-reaching implications for the mathematics and 
science education system. Also, we sought to discover what leaders identified and addressed as 
particular barriers in their current systems that could limit the impact of the other changes they 
pursued (Banathy & Jenks, 1990; Orians, 1990). In both cases, our investigation probed not only 
what critical targets of change SSI leaders identified, but also why they saw those as critical 
targets, how they expected to change them, what the SSI was able to accomplish in those areas, 
and whether those targets, or others, turned out to be critical in the initiative. Moreover, we 
adopted Elmore’s (1996) very useful distinction between “scaling-up” and “going to scale.” 

With regard to critical targets of change, the strategic thinking behind the SSIs may be 
illuminated by the intention to scale-up certain changes — reaching more districts, schools, 
teachers and students over time with particular interventions — versus the intention to use some 
changes to go to scale with the reform — altering the system so that all districts, schools, teachers, 
and students are deeply influenced by changes in the expectations and support systems that 
become a part of the normal modes of operation of the system. 

Trade-Offs and Balance 

The trade-offs that leaders make and the balances that they strike among short- and long-term 
goals are an important aspect of their strategic thinking about systems change, especially in terms 
of sustaining changes (Dovers, 1990; Farrell & Hart, 1998, Niu, Lu, & Khan, 1993). Given the 
charge to change an entire state’s mathematics and science education system, the SSIs had many 
possible targets to choose from when they designed and implemented their initiatives. In 
addition to identifying critical targets for change, leaders of the SSIs had to consider what could 
and could not be done within the resources they had or could obtain. Also, since many SSIs 
engaged in direct work with districts, schools, and teachers, but none could do so with all, leaders 
had to make choices that they reasoned would give them the best pay-off. How leaders weighed 
trade-offs among different courses of action was a useful line of investigation for the study of 
strategic thinking. Another important trade-off the SSI leaders had to consider, and one over 
which the context and history of the state often had considerable influence, was between 
centralized guidance and local decision-making. Although the theoretical basis for systemic 
education reform (Clune, 1993; Smith & O’ Day, 1991) provided some direction on this trade-off, 
leaders also had to consider how much centralized guidance districts and schools expected or 
would tolerate; how much capacity for local decision-making existed and whether it was likely to 
be aligned with the reform vision; and how quality control over interventions and policy 
implementation would occur at the local level. 
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The balance of short- and long-term goals is also a particularly salient feature of systemic reform, 
especially in light of the goal of sustaining the benefits of the reform and the reform itself. 
Researchers of sustainable national development programs (Comforth, 1999; Doryan, 1993) 
articulate a framework for balancing demonstrable productivity, including early attention to the 
desired outcomes of development, with human capacity building and economic assurances in the 
long-term. For a variety of reasons, the SSIs attended to some short-term objectives, often with 
clear benefits and high visibility, along with their pursuit of long-term goals for deep structural 
change in the education system. Striking the right balance between the short- and long-term aims 
of the reform is a considerable challenge for leaders of large, complex reforms such as the SSIs. 
For example, lacking some demonstration of success in the short-term on which to build support, 
capacity building or infrastructure development efforts directed toward long-term goals of deep 
structural change in the education system may be unlikely to survive the time period they require 
to produce impacts. At the same time, too much attention to direct interventions for short-term 
gains without consideration of the time, resources, and concerted effort required to attain deep 
structural changes may result in only short-lived reforms. 

Flexibility to Deal with Uncertainty 

In his book, The Renewal Factor , Waterman (1987) explored how organizations often take a 
linear approach to a process that is constantly changing. He calls those organizations that can 
adapt to changing circumstances “renewing organizations.” As he wrote, “Most companies try to 
overlap a rational, linear, deterministic technique which they call strategy on an underlying 
process that is random, full of surprises. . . . The rational plan will not work unless it contains 
sufficient flexibility and elbow room to ‘go with the flow.’ Strategic methods must be able to fit 
the unpredictable forces at work” (p. 8). The degree of uncertainty the SSIs faced was palpable. 
The state contexts in which they operated were constantly changing in terms of leadership, 
policies, and priorities, or at least the possibility of such changes was always looming. Our study 
of strategic thinking in the SSIs investigated how leaders anticipated and prepared for certain 
events as a means to reduce uncertainty, and how they handled events that came as surprises to 
them, whether in their favor or not. 

Politics 

Both internal and external politics of organizations can be prime drivers of strategic thinking. 
Goldsmith (1996) argues that key decisions often result as much from internal politics as from 
impartial analysis. As he describes it, interest groups within all organizations attempt to impose 
their ideas, and many real decisions are made away from formal meetings, in informal 
bargaining, negotiating, and coalition-building among these interest groups. McMillan (1978) 
focused on the identification of, and reaction to, external political challenges and opportunities as 
a central tenet of strategic thinking. Cameron (1981) and Jobson and Schneck (1982) 
emphasized the idea that different organizations pay attention to different constituents as a result 
of their strategic thinking about whose interests matter most to the organization. Koteen (1989) 
captured a similar notion in an analysis of non-profit organizations, arguing that strategic thinkers 
consider the needs and interests of both those paying for services, and those receiving services; 
these are often not the same people, and consequently, are often not the same needs and interests. 
Comforth (1999) and Doryan (1993), in their writings on national development programs, 
concluded that reforms aimed at sustainable systems change rested not only on the ability to 



Horizon Research, Inc. 



11 



March 2003 




produce intended outcomes and to enhance capacity and resources to expand and continue their 
services, but also on the political acceptability of the reforms’ processes and outcomes. 

In the SSIs, leaders pulled important partners together within their states. The identities and 
interests of those partners may have exerted considerable influence on the design and 
implementation of the initiative. Looking outward, the SSI leaders identified constituents or 
markets for the services of the initiative, attempting to serve their needs. Also, they attempted to 
communicate with key stakeholders whose support, or at least lack of opposition, was vital to 
moving the initiative forward. Our study attended to both internal and external political 
considerations. Internally, in particular, we investigated how complementary or conflicting 
interests of those within the SSI influenced the strategic thinking behind the initiative. 

Externally, we sought to understand who the SSI intended to serve as its main audiences, and 
whose attention, interest, and support the SSI attempted to cultivate. 



Methodology 



Research Question 

The study of the impact of the Statewide Systemic Initiatives Program: Lessons Learned was 
originally conceived to address the question: What lessons have been learned about designing, 
implementing, evaluating, and supporting statewide systemic reform? As the study progressed, 
the research focused in two specific areas. Lirst, we investigated how the SSIs’ leaders thought 
about design, implementation, evaluation, and support of their SSI in terms of: 

1. What strategies the SSI would employ for: 

a. Increasing the capacity of teachers, schools, and districts to undertake 
reform; 

b. Building infrastructure for implementing reform; 

c. Gaining public, political, and professional support for reform and avoiding 
opposition; and 

d. Aligning policies in support of reform; 

2. How the reforms initiated by the SSI would address the need for improved 
excellence and equity in the teaching and learning of mathematics and science 
throughout the education system; 

3. How the reforms initiated by the SSI would reach all districts, schools, teachers, 
and students in the state; and 

4. How the reforms initiated by the SSI would be sustained beyond the funding of 
the SSI. 

Second, we investigated how the SSIs’ leaders thought about the appropriate role of the SSI 
within the state education system, particularly in terms of: 
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5. What the SSI should be positioned to do given its fiscal, material, organizational, 
and human resources and where its services were best and most necessarily 
applied; 

6. How the technological, economic, political, social, and historical context of the 
state would influence what the SSI could and should do; and 

7. How the SSI should match its services to the needs of the education system and to 
the opportunities that were afforded or created. 

The investigation of the research question included study of both the initial plans for the SSIs and 
the refinements and mid-course corrections as the initiatives unfolded. Also important in 
addressing this question was how leaders viewed the connection between what the SSIs would do 
and what else was happening in mathematics and science education in the state, including both 
state policies/programs and other projects. In some cases, the SSI itself might take on many or 
all aspects of systemic reform, while in others the SSI might be focused on one piece of a larger 
systemic change effort in the state. 

Sample and Data Sources 

A total of 25 states and the Commonwealth of Puerto Rico received SSI awards. Four states did 
not finish Phase I and were excluded from the study because they were deemed by NSF not to be 
in compliance with the parameters of the SSI program. Leaders in one of the states that 
completed Phase I requested to be excluded from the study. The remaining 21 SSIs were all 
included in the study to varying degrees. 

In order to gain understanding about the research question, two principal data sources in the SSIs 
were consulted: documents, and people who served in leadership roles in the SSIs. Documents 
included both internal documents produced by the SSIs and externally authored reports on the 
SSIs. These documents were used as data sources primarily because they captured information at 
specific points in time, providing a chronology of events in the SSI and a perspective on events 
close to the time of their occurrence. The documents were most useful for understanding the 
historical and contextual environment of the states, the basic chronology of the initiatives, and 
the components of the initiatives. 

Internal documents examined for the study included the funded proposals for the first five years 
of the SSI program in Phase I, funded and non-funded proposals for the continuation of the SSI 
program in Phase II, responses to reviewers’ questions prior to funding decisions on Phase I and 
Phase II proposals, mid-term and final SSI reports to NSF, and individual SSI evaluation reports. 
Internal documents had to be requested from the states and were collected from 18 of them. We 
were unable to obtain some of the documents, leaving us with incomplete data on some SSIs. 

The complete set of internal documents was collected and reviewed for 13 SSIs. 

The main external documents reviewed for this study were the case studies and other program 
evaluation reports produced by SRI International as the national evaluator of the SSI program, 
and the state monitoring reports written by staff and consultants of Abt Associates, Inc. as the 
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national monitoring agency for the SSI program. External documents were included from all 21 
states. 

For more intensive data collection, the sample had to be narrowed somewhat due to resource 
availability for the study. In order to continue following the best trail of information, the 
researchers conducted interviews and/or site visits for those 13 SSIs for which full document 
review was undertaken. Some bias may have been introduced into the sample due to this choice. 
The results of the study should be interpreted with this caution in mind. 

The purpose of the interviews and site visits was to obtain further information on the chronology 
and components of the SSIs, and more importantly to gain insight into the rationale behind the 
design, refinements, and mid-course corrections of the SSIs; the progress of the initiatives over 
time; the sustained aspects of the initiatives beyond NSF funding; and reflections on the design, 
implementation, evaluation and impact of the SSIs. In 7 of these 13 states, telephone interviews 
were planned with two primary SSI leaders — the most heavily involved Principal Investigators, 
Co-Principal Investigators, or Project Directors. In the remaining six states, three-day site visits 
were planned that included face-to-face interviews of 6 or 7 SSI leaders in each state. 

This final report was reviewed by at least one leader from each SSI that was named in the report. 
Only those states that were the subject of a full document review and telephone interviews or a 
site visit are named in the report, as the research team’s interpretations and analyses were only 
considered complete in those states. The Case Reports were reviewed by at least two leaders in 
the respective SSI. These reviews led to some additional data collection in the form of telephone 
calls and e-mail messages that was used to clarify and update information in the manuscripts. 

Data Collection, Instrumentation, and Analysis 

Data collection, instrumentation, and analysis were conducted in steps intended to provide the 
research team periodic opportunities for reflection on information and issues in order to inform 
future steps in data collection and analysis. 

As noted earlier, data sources for this study included internal documents, external reports, and 
leaders of the SSIs. Instruments for data collection included document review and interview 
protocols. Instruments for analysis included several coding schemes that evolved as a part of the 
analysis, and periodic writing tasks to appraise the reliability and utility of coding schemes. 

Data Source: External Documents 

The first data sources obtained and analyzed were external documents. These documents 
provided a good starting point for the study because they offered the most objective information 
that was systematically collected on the SSIs at the state level during Phase I of the program. 
Documents included the SRI International evaluation reports (listed in Appendix A), and the Abt 
Associates monitoring reports (listed in Appendix B). 

Data Analysis: Coding of Chronologies and Components 

External documents were initially reviewed and coded for chronology; key components, leaders, 
events, and outcomes; and important contextual considerations. The coding scheme for 
chronologies and components is presented in Appendix C. The main product of analysis from 
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this first round of coding was a summary matrix for each SSI describing the context, 
organization, components, and impacts of the SSI over time. From information compiled from 
this initial round of coding and analysis, a data collection plan for internal documents and a 
second coding scheme were developed. 

Data Source: Internal Documents 

The plan for collecting internal documents emerged from the coding of chronology and 
components. The plan identified a set of documents that spanned design, implementation, and 
evaluation of Phase I of the SSI program and initial design for Phase II of the SSI program. The 
documents specified in this plan were selected because they would be likely to describe the initial 
SSI plans and the progress of the initiatives over time; to identify and provide a rationale for the 
SSI design and changes to the plan over time; to document and report interim impacts of the SSI; 
and to indicate future direction for the initiative. Nearly all of the internal documents specified 
were produced as required for NSF program purposes. The primary audience for nearly all of 
these documents was the funder, which undoubtedly limited the range of perspectives and 
information available from this data source. However, this limitation was acceptable because the 
set of documents specified was known to exist for all SSI states, an important consideration for 
assuring comparable data across sites, and the research team had already planned site visits and 
interviews that would follow the document review. 

Data Analysis: Coding of Strategic Planning and Decision-making 

The purpose of applying a second coding scheme in the study was to move the analysis beyond 
an understanding of the structures and goals of the SSIs, and the sequences of activities each SSI 
planned to accomplish those goals, and toward an understanding of the reasoning behind the 
structures, goals, and activities. Also, we sought to understand the reasoning that led to changes 
in structures, goals, and activities as the initiatives progressed. 

A coding scheme that focused on the strategic planning and decision-making of the initiatives, as 
well as the influence of historical and contextual considerations on the strategies (see Appendix 
D), was applied to analyze internal documents and to add a second layer of analysis to external 
documents. Strategies described for the design, implementation, and evaluation of each initiative 
were coded. Changes and refinements of strategies in response to changes in the context, 
leadership, or progress of the initiatives were captured. The main product of the second round of 
coding was an analytic summary of evidence for each SSI organized by the coding scheme. The 
second round of coding also informed the development of a general interview protocol and 
refinements and tailoring of the protocol for particular states and particular leaders to be 
interviewed. 

Data Collection: Site Visits and Telephone Interviews 

As a part of the document review process, key leaders in each SSI were identified for interviews. 
The 13 SSIs for which full external and internal document review was completed were divided 
into two groups: 6 site visit SSIs and 7 telephone interview SSIs. The identification of SSIs for 
site visits was conducted purposefully in order to provide the most in depth data collection of the 
study on a set of SSIs that varied on a number of dimensions. These dimensions were identified 
in the document reviews as characteristics of states that were potentially important in the design, 
implementation, and evaluation of the SSIs. The dimensions of variability included 
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demographics, history of reform, locus of educational control, policy context for systemic 
reform, comprehensiveness of the SSI plan, and organizational home of the SSI. It was also 
deemed important to include in the set of site visit states: SSIs that received Phase II funding, 
SSIs that were generally considered successful in Phase I but that did not receive Phase II 
funding, and states that were considered to have mixed results in Phase I and did not receive 
Phase II funding. 

For the states identified for telephone interviews, we planned to contact two key SSI leaders to be 
interviewed by a project researcher using a tailored version of the study interview protocol. (See 
Appendix F.) In five of these SSIs, two leaders were interviewed; in one, three were 
interviewed; and in one, one leader was interviewed. Interviews were conducted between June 
2001 and April 2002. Each interview lasted approximately 75 minutes. Follow-up questions 
were handled with brief additional telephone calls or e-mail. 

For the states identified for site visits, we planned to contact two key SSI leaders in each state to 
schedule the site visit and to identify a list of other SSI leaders as potential interviewees. This 
list was compared to the list generated as a part of the document review to identify additional 
leaders to be interviewed as a part of the site visit. The site visits were completed for four of 
these states, with a few of the interviews occurring by telephone due to scheduling constraints. 
Two of the states were not visited due to travel constraints surrounding the events of September 
11, 2001. For these two states, the full slate of interviews was conducted by telephone. The site 
visits and interviews took place between June and December 2001. 

Data Analysis: Coding of Strategic Planning and Decision-making and Developing the Case 
Reports 

Data from the telephone interviews were incorporated into the analytic summaries for these SSIs 
using the coding of strategic planning and decision-making. For SSIs in the site visit group, the 
analytic summary and interview data were used to write an SSI case report. (See Appendices G- 
L.) The case reports were intended to be flexible narratives so that the story of each SSI and 
issues of particular pertinence would be in the forefront, with common issues arising from the 
coding of strategic planning and decision-making embedded in the narratives. 

Data Analysis: Coding of Key Challenges for Large-Scale Systemic Reform and Developing 
the Cross-Case Report 

During the collection and analysis of data from telephone interviews and site visits, including the 
drafting of case reports, the research team examined the analytic summaries and case reports to 
identify emerging themes across states. From this examination, a new coding scheme was 
developed around key challenges for large-scale systemic reform (see Appendix E). These key 
challenges were not all encountered, recognized, or addressed in every SSI, but were commonly 
evident and identified as critical in many of the SSIs. The analytic summaries and case reports 
were coded using the new coding scheme organized around these key challenges. This process 
coding and analysis was conducted in order to juxtapose the particular strategies of the SSIs 
against the key challenges addressed by those strategies. 

Following the coding of key challenges for large-scale systemic reform, the case reports were 
revised to reflect new understandings about how strategic planning and decision-making were 
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employed to address key challenges within those states. Also, a cross-case analysis was 
conducted by aggregating coded data across states using the key challenges coding scheme. The 
product of the cross case analysis is presented in the subsequent chapters of this volume. 

Periodic Reviews 

An integral part of the data collection, analysis, and writing was periodic review of data 
collection plans, data collection instruments, coding schemes, interim written products, and final 
written products. In addition to meetings of the research team to make decisions throughout the 
study, reviews by project advisors, our WCER research partners, and key SSI leaders were 
incorporated as an integral piece of the project design. 

The project held annual advisory board meetings. At these meetings advisors were apprised of 
the current status of the project and key upcoming decisions. Advisors were also asked to review 
data collection plans, data collection instruments, coding schemes, and interim and draft 
products. The project held additional working meetings of the WCER and HRI researchers. 
These meetings provided opportunity for the WCER research team to review plans, instruments, 
analysis, and interim products. The recommendations and reviews of the advisors and research 
partners proved invaluable in many of the key decisions throughout the life of the study. 

Reviews of draft reports were also solicited from key leaders in the SSIs. Draft versions of the 
case reports were sent to the two initially-identified leaders in the subject SSIs. A draft version 
of the cross site report was sent to one leader in each SSI that was named in the report. In both 
cases, these leaders were asked to review the case reports and cross-case report for accuracy, 
clarity, and political sensitivity. Inaccuracies were corrected and descriptions were clarified 
through these reviews. Politically-sensitive presentations of material were modified either to 
present the information anonymously or to provide an additional balancing viewpoint. 
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Part II: Findings 



Systemic change was a new concept when the SSIs began their work. The SSI solicitation 
described a vision consistent with the emerging national standards for mathematics and science 
education. In response, SSI proposals described plans for activities that were based on that 
vision, or their interpretation of how the national standards vision would play out in their state. 
Although the solicitation described several of the “components” of the system that would need to 
be aligned with this vision, there was no road map for which components to choose as a primary 
focus, nor how best to go from where a state was starting to the end destination of excellent 
science and mathematics education for all students. Each SSI embarked on the set of activities 
that NSF reviewers, themselves new to systemic reform, had deemed of sufficient quality, depth, 
and breadth to transform mathematics and science education in the state. 

The profiles of priority needs described in the proposals of the awarded SSIs were quite similar 
to one another and to the needs in mathematics and science education nationally. Problematic 
areas presented in the proposals typically included the lack of teacher mathematics/science 
content knowledge at the elementary level; resistance to pedagogical reform at the high school 
level; and a mixture of the two problems in the middle grades, with few teachers having been 
prepared specifically for mathematics/science teaching at the middle school level. The proposals 
also typically raised a number of equity concerns, providing evidence of differential opportunity 
and performance gaps in their states based on race, gender, socioeconomic status and/or 
geography. Many described problems at the higher education level as well, indicating that arts 
and science departments and education departments operated in isolation from one another, and 
that pre-service education programs suffered as a result. 

Not only was the vision of quality mathematics and science education underlying the SSI 
proposals remarkably alike, so too were their goals and objectives. As specified in the 
solicitation, all embraced a vision of mathematics and science education based on emerging 
national standards, with all students actively engaged in developing understanding of powerful 
mathematics and science concepts rather than focusing heavily on mathematics algorithms and 
science vocabulary. And nearly every proposal sought to achieve that vision by increasing 
teacher knowledge and skills, thereby improving classroom practice in order to raise student 
achievement levels and narrow the gaps among various demographic groups. 

Although the vision, needs in relation to the vision, and goals of the initiatives were very similar, 
the SSI proposals described a variety of efforts to achieve their goals. The planned activities 
typically targeted one or more documented needs, and the proposals described plans to reach 
large numbers of teachers in order to address those needs. In many cases, the sense one gets in 
reading the SSI proposals is of major projects that would try to do as much good as they could 
until the grant ended, hoping that the states would continue to fund at least some of the SSI 
activities. Using the distinction Elmore (1996) has described, in essence, the SSIs approached 
systemic reform with a “scaling up” strategy to reach ever larger numbers of teachers, rather than 
a “going to scale” strategy, where the reforms would become an integral part of the system. 
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From our vantage point in 2003, it is clear that to be successful in changing the mathematics/ 
science education “system,” each SSI needed a strategy that would lead to a shared vision of 
reform. This vision requires not just agreement on what quality mathematics and science 
education looks like, but also agreement on a plan for how to accomplish substantial reform 
during the funded period and to ensure that the improvement process would continue beyond 
that. In this context, SSIs would need to accomplish two key tasks. First, the SSIs would have to 
establish the kinds of interventions needed to increase the knowledge and skills of the 
participating teachers, administrators, and students. Second, the SSIs would have to foster the 
professional, political, and public support to put the needed interventions into place on a large 
scale. 

We use the term technical strategy to describe how the SSIs thought about, and went about the 
work of, establishing the kinds of interventions needed to bring about changes in teaching and 
learning that result in improved and more equitable student achievement. Some SSIs chose to 
develop and provide those interventions directly; others leveraged service providers to deliver 
those interventions. Nearly all SSIs thought about and worked on reform interventions for in- 
service teachers, and some also worked on interventions in areas such as pre-service education, 
or curriculum materials. The work of building infrastructure and capacity to deliver interventions 
is a part of an SSI’s technical strategy. The main aim of the technical strategy was developing 
human and organizational capacities for reform. 

We use the term political strategy to describe how the SSIs thought about, and went about the 
work of, establishing a supportive context for reform. The needed elements of a supportive 
context included formal guidance and accountability policies aligned with the vision; allocation 
of resources to scale-up and continue reform efforts; and a culture of professional, political, and 
public understanding and commitment to the vision. Some SSIs began with policy work and 
some organized deliberate outreach efforts; others put less emphasis in these areas or held off on 
these efforts until they had established the work of their technical strategies. The main aim of the 
political strategy was fostering the professional, political, and public will for reform. 

Our definition of a strategic systemic reform initiative includes a coordinated technical strategy 
and political strategy that together establish the capacity or means for reform, as well as the will 
or opportunity to accomplish a vision of high quality mathematics and science teaching and 
learning for all. We do not advocate a narrow definition of the particular technical or political 
strategy, nor do we prescribe a particular relationship between the two. We do suggest that the 
two must be tailored to the state history and context; must be designed carefully to meet common 
challenges to reform; must be monitored and refined to assure that high quality is established and 
maintained; and must be mutually reinforcing to support large-scale, sustained system change. 

To add to the challenge, if the systemic reform efforts are dependent on an external funding 
source, as was the case in the SSIs, they must keep the funders informed and satisfied as well. 

In the following chapters, we discuss the technical strategies employed by the SSIs in trying to 
improve mathematics and science education, the political strategies they used in gaining support 
for reform within their states, and how they managed interactions with the funder as the 
initiatives unfolded. 
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Technical Strategies for Systemic Reform 



“To have a vision is one thing. To be able to visualize how you are going to 
enact it would be something different.” (SSI Principal Investigator) 



“We just make sure that our work is of the highest quality. When we make a 
mistake we work doubly hard to correct it, so that the next time out we’re back 
to the same quality level.” (SSI Project Director) 



Through their technical strategy, SSI leaders specified the activities that they believed, if 
implemented well, would transform teaching and learning consistent with their reform vision. A 
complete technical strategy would address a common set of key challenges in the state for 
building the necessary capacity for reform. 

First, interventions needed to be defined around an understanding of the quality and intensity of 
effort needed to have a substantial effect on teaching and learning. The interventions could be 
used to operationalize the vision — to show people what the quality mathematics and science 
education described in state and national standards documents actually looked like on the ground, 
involving real teachers and real kids, in typical schools and districts with less than ideal 
resources. 

Second, monitoring, evaluation, and/or research needed to be used to assure that the interventions 
were up to the needed quality and intensity to change teaching and learning; to refine the 
intervention, if needed; and to demonstrate that the interventions actually “worked.” That is, the 
SSI needed to be sure that teachers who participated in SSI-supported professional development 
and/or used SSI-endorsed materials changed their classroom practice consistent with the reform 
vision, and that their students learned more, and more powerful, mathematics and science as a 
result. 

Third, these “existence proofs” needed to be used to demonstrate more broadly not only what the 
vision looked like in action, but also that it was indeed feasible to implement the reforms in a 
variety of contexts in the state. Moreover, a powerful existence proof should also establish that 
teachers and students benefited from the interventions in clear and measurable ways. The early 
interventions could be used as a training ground for teacher leaders and other change agents, 
providing an opportunity to increase capacity within the state to deliver services on a much 
broader scale as the initiative proceeded, and beyond the funded period. By doing so, the SSI 
would help ensure that increased resources would result not just in more reform “activity,” but in 
high quality interventions and improved outcomes. 

Finally, the interventions had to be scalable. It was not enough to show that well-designed 
interventions delivered to a fairly small group of willing participants produced the intended 
changes. It was also important to demonstrate that the reform interventions could be delivered 
more broadly, maintaining quality and producing comparable outcomes within a cost-efficient 
use of resources. 
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Our criteria for a sound technical strategy, then, are that the strategy: 

• Operationalizes the reform vision through interventions; 

• Monitors and refines the interventions, and provides evidence that they result in 
improved classroom practice and improved student outcomes; 

• Increases capacity within the state to scale up the reform efforts; and 

• Provides evidence that quality and impact are maintained during scale-up. 

Although it might be necessary for an SSI to work with a fairly large number of schools or 
teachers in order to provide evidence that the reforms could be implemented successfully in a 
variety of contexts in the state, this line of reasoning suggests that working directly with schools 
or teachers during the funded period is best thought of as a means to increase the likelihood that 
broad reform will be accomplished, rather than as an end in itself. This chapter examines the 
nature of the SSI activity against this backdrop, examining the extent to which, and the reasoning 
behind, states’ attending to various components of the suggested definition of a sound technical 
strategy. 

Operationalizing the reform vision through interventions 

SSIs had a series of decisions to make about the early reform interventions that would both 
define the initiative and operationalize the reform vision: whether to start with mathematics, 
science, or both; whether to focus initially on a particular grade range, or work across the entire 
K-12 spectrum; whether to work with all districts, select a set of districts that were broadly 
representative of the state, or give priority to high needs districts; whether to conduct the 
interventions centrally, or from a regional infrastructure; whether to specify the interventions in 
detail, or set parameters for the interventions and let existing service providers develop and 
deliver them; when to tackle pre-service education reform; etc. The following sections describe 
some of the decisions SSIs made, and the reasoning behind those decisions. 

Deciding Where to Focus the SSI Efforts 

The SSI solicitation emphasized the importance of involving people with expertise in 
mathematics and science education in the design and implementation of the initiatives, and many 
of the SSIs were headed by people with considerable background in the field. In some cases the 
Pis’ research interests, and/or their understanding of the barriers and opportunities in 
mathematics and science education generally, were the driving force behind the entry point for 
the SSI interventions. 

The PI of the Georgia SSI, an expert in middle school science education, explained the reasoning 
behind that SSI’s choice of the middle grades as an area of intensive initial focus, coupled with 
components targeting a broader scope of the system. 

We really thought that was the critical area to make change. Elementary school is such 
a difficult [challenge] and it would be so labor-intensive to try to make changes there 
because of the lack of substantive knowledge among teachers. We felt that in Georgia 
that we had a middle school certificate, so we had people who wanted to be there. We 
had specifically science-trained middle school teachers. We didn ’t think that they knew 
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enough, but they certainly were on the road towards knowing enough. And we felt that in 
contrast, secondary was a difficult animal to change. There are not a lot of good 
curriculum materials out there. [Teachers] tend to be entrenched and more traditional. 
So we just thought that this was a good place to start, and then that we could filter up 
and down through the system. 

Many of the SSIs began their reforms at the elementary or middle school grades, citing both the 
pressing needs and the likely opportunities at those levels. Said a PI of the New Jersey SSI: 

We felt the professional development void was greatest in the K-8 level. We only had a 
K-8 certificate, not even a middle school certificate. This was a pressing need. We also 
felt we would have some difficulty making a reed impact at the high school level if we 
dicln 7 have impact first at the K-8 level. We also felt that elementary and middle schools 
would be more receptive because they dicln 7 have entrenched departmentalization. 

In interviews, SSI leaders indicated that when their reforms were aimed at the entire K-12 
spectrum, they quickly began to “run out” of secondary mathematics and science teachers who 
were both amenable to reform and willing to devote the time required to participate in 
professional development programs. Explained a faculty member who taught both elementary 
and secondary level in-service education courses as part of the Arkansas SSI: 

When lam working with elementary teachers, they’re so hungry for “Can you give me 
one more way to explcdn this, one more visued image, one more tactile thing?” Then as 
you move up to middle school, there ’s some, “Well I can see some advantage to doing 
some of this. ” And then you get up to high school, “ Well that’s nice, but it would take up 
a whole day and I have cdl this curriculum to cover. ” 

Equity concerns figured prominently in many of the SSI plans, in some cases dictating the choice 
of grade range to target. For example, a PI of the Ohio SSI explained their decision to focus on 
the middle grades, “That was really an equity decision because we wanted to reach out 
particularly to the urban minority kids. Tracking begins in grade 9, so we figured all kids were 
still taking math and science in middle grades.” 

A major goal of the Connecticut SSI was to “devote significant resources to increased 
educational equity and access for the state’s K-12 urban minority students.” The project’s 
Coordinating Board was to have a significant minority representation and major funds were to be 
spent on equity issues. One of the initiative’s primary purposes was to “advocate for minority 
equity and access” in education, so preference for participation in SSI activities would be given 
to the “priority” districts, those that had a high number of economically disadvantaged and 
academically underachieving minority students. 

Similarly, all teacher education programs developed by the Georgia SSI would include a focus on 
females, minorities, and disabled students. A program to attract minorities to the teaching 
profession would target early identification, recruitment, and retention of pre-college minority 
students. 
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In Maine, Community Action Teams would be charged with identifying the needs of underserved 
groups of students in their communities and developing programs to address those needs. 
Although the SSI would leave decisions about which programs were most appropriate for each 
community up to local leaders, apprenticeship programs with businesses and other local partners 
were to be a feature of the services to underserved students in all communities. 

Recognizing that teachers in the most disadvantaged districts are often the least well-prepared, 
the Michigan SSI planned to improve professional development offered through the state’s 
Mathematics and Science Centers, and to learn more about the needs of teachers and students in 
targeted high needs urban and rural districts, so they could subsequently push for policies that 
would be supportive of high needs districts in the state as a whole. 

Equity continued to be a key focus as the SSIs moved from planning into implementation. In 
Puerto Rico, for example, the fact that the SSI provided professional development at school sites 
throughout the Commonwealth helped a great deal with the equity focus. Said one of the Co-PIs: 

What we most tried to do was level the playing field in the schools, for example, in 
schools that had no materials, we provided them. And training teachers— historically, 
most of the teachers who were trained were in the metropolitan areas. So in leveling the 
playing field in teacher training, we took the professional development to the whole 
island, so that teachers in rural areas would have access to the same quality training as 
teachers in the metropolitan areas. 

In the Ohio SSI, individual regions had responsibility for selecting the teacher leaders and the 
mathematics/science educators who would be trained to provide professional development. To 
correct an under-representation of minorities among these professional development providers, 
the Pis offered an incentive, telling the regional directors, “if you will select a minority, we will 
give you an extra math/science educator/teacher leader slot”, which resulted in a substantial 
increase in the number of minority professional development providers. 

Although not a prominent feature of the SSI proposals, as the initiatives unfolded some of the 
SSIs linked up with other groups who had equity as a major focus in order to further the SSIs’ 
mathematics and science education reform agenda. The PI of the Texas SSI described the 
importance of creating these linkages: 

Math people and science people may care about equity, but what they know how to do is 
math and science. Title I folks, or people who work on equity, are driven by the needs of 
particular kids and providing the social and education environments; they don 7 have 
access to rich content. The only way in which we can really build bridges is by 
integrating the organizations and it’s massively complicated because they have 
completely different cultures. We also had Head Start here for a few years. So you have 
Federal; you have State agency things; you have non-profit things and building local 
management; you have university culture. So the idea is to create, by integrating 
management and blending cultures, you try to create the human networks that allow 
equity to be achieved through these math and science efforts. 
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He concluded that the results were worth the effort, but that “it’s been very difficult to manage.” 
Starting with What They Knew How to Do 

Given that systemic reform was a relatively new and somewhat overwhelming notion when the 
SSIs began their work, it is not surprising that many of the SSI leaders chose to start with a 
“familiar” intervention, focusing their efforts on something that was already underway in the 
state, with the idea of increasing its scale or expanding it to address both mathematics and 
science and/or a wider grade range. 

For example, the Vermont SSI based part of its plan on models in use in teacher enhancement 
projects in the state — intensive summer institutes and follow-up sessions that would build 
teachers’ ability to implement inquiry-based instruction in their classrooms. The expertise to 
conduct the institutes was to come from the people who had been running their efforts 
successfully, college and university science and mathematics educators who were well-respected 
by teachers and knowledgeable about how to structure professional development sessions. 

As another example, a major professional development effort was already underway in Arkansas, 
a “Mathematics Crusade” supported through a combination of Eisenhower grants, state and local 
school district resources, and private funds that had provided professional development to a large 
number of middle school mathematics teachers. Said one member of the Arkansas SSI planning 
team: 



I think we had already moved in terms of doing some things in mathematics and had the 
Math Crusade there and it was beginning to get a toehold. I don ’t think there was any 
question [that this] is the direction that we need to go in. And the fact that we sort of had 
this thing put together, / think gave us the impetus to go ahead and bring science along in 
a like fashion. 

In an interesting variation on this theme, sometimes the SSI was designed around a program that 
one of the key players had been involved in, albeit in another state. For example, the South 
Carolina SSI’s Curriculum Leadership Institutes were modeled after the California Subject 
Matter Projects which the PI had helped administer. 

In other cases, the plan was to design new programs, but to begin them with people and places 
that were already active in reform, planning to establish models and radiate out from those. In 
Nebraska, some of the state’s strongest secondary mathematics teachers had participated in 
extensive teacher enhancement activities through the Nebraska Mathematics Scholars project. 
With a strong cadre of lead teachers and school contacts across the state, SSI leaders saw the 
initiative as an opportunity to build on these efforts and “take it to the next level.” In addition, 
Nebraska had already invested considerable resources in establishing a viable distance learning 
strategy; SSI leaders planned to build on this system, connecting K-12 educators through 
electronic networks, and offering courses and resources to enhance curriculum and professional 
development opportunities for teachers across the state. 

Two of the Professional Development Schools involved in the Georgia SSI had been involved 
with the American Association for the Advancement of Science’s Project 2061. Similarly, 
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Delaware and Maine both began their SSIs working with schools that had been involved in the 
Coalition of Essential Schools’ Re: Learning Network. One leader in Delaware stated why this 
choice was natural for the SSI, “We had engaged a fair number of schools who were being 
empowered to look at a new way of teaching and learning. Several of them had focused on math 
and science teachers.” Although the Re: Learning Network did not have a mathematics or 
science focus, per se, the SSI worked to provide expertise and to refocus the schools on 
mathematics and science. 

Going statewide with something they knew how to do on a small scale, typically in one subject in 
a narrow grade range, with reform-oriented teachers and schools, was in itself a major challenge. 
Project staff needed to broaden their efforts from working with people who shared their vision 
and volunteered to participate, to figuring out how to engage the “wait and see” teachers, not to 
mention the outright critics of reform. As it turned out, SSIs were more successful in 
operationalizing the reform in some areas than in others, typically having more success in the 
subject/grade ranges they had started with than in the expansion areas. Pis reported particular 
difficulty with reform at the high school level. Lor example, “After that initial year we 
broadened it [from the middle school level]. . . .The elementary schools seemed to be more 
receptive. I don’t think the high schools were ever as receptive.” 

The attempt to serve large numbers of teachers of varying predispositions to reform, in both 
science and mathematics, and across the entire K-12 spectrum, appears to have acted against the 
SSIs using the interventions strategically as we have defined it. The amount of energy it took 
simply to get large-scale interventions up and running distracted the SSI leaders from important 
strategic tasks. Tasks that often received less attention included fine tuning the interventions to 
assure that they could maintain high quality with the most efficient use of resources, and focusing 
on using the interventions to systematically collect evidence of the effectiveness of the reforms in 
a variety of contexts throughout the state, so that the SSI could generate support for additional 
reform interventions. 

Coordinating with Existing Reform Efforts 

The need to coordinate their work with other reforms underway in the state added to the 
challenge. The SSI solicitation explicitly requested that the initiatives coordinate with other 
efforts in the state, and proposals typically listed a number of reform initiatives underway, 
especially if they were funded by NSL, and described how they would either be coordinated with 
or integrated into the plan for systemic reform. As the SSIs unfolded, it often transpired that 
these efforts continued to operate independently. In interviews, SSI leaders sometimes spoke of 
their concerns about the quality of some of these initiatives, and their frustration that they could 
not do much about it. One PI ruefully recalled: 

It wasn ’t a unified effort of how we could come together to solve a problem, [but rather] 
“It’s our little thing and that’s all we have to do and we’re not really involved in the 
bigger problem. ”... Because of that initial mistake, it was a battle from then on in. . . . 
The way that the proposal was written is that we got groups together on numerous 
occasions and we’d say, “All right, write us a little proposal of what you’ d like to do” 
without any guidance or general sense of direction. So we just kept trying to fix that all 
the way through. 
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In some cases, these pre-existing, peripheral initiatives that were intended to receive some funds 
from the SSI did not last very long. In other cases, SSI leaders could not remember what had 
happened to the separate initiatives that were supposed to complement the major SSI initiatives; 
they “guessed” that the projects had simply ended when their funding ran out. Interviewees 
typically could not recall decisions to end them, or cite reasons for their demise; rather the press 
of keeping the main efforts moving forward kept them from attending to these supplementary 
programs, and they just disappeared. 

A Trend Toward Regionalization 

Several of the SSIs began with plans to develop (or increase the capacity of existing) regional 
entities to deliver professional development, technical assistance, and implementation support. 
The idea was that regional groups would have a better awareness of needs and resources in their 
area, and could better devise strategies for linking teachers with colleges and universities, 
informal science institutions, and community-based groups. 

A number of other SSIs that had originally planned to do much of the intervention centrally 
wound up focusing more of their resources on regional service delivery as the initiatives 
progressed. In Nebraska, for example, the Regional Coalitions increasingly undertook a wide 
array of activities: conducting needs assessments; publicizing and supporting professional 
development activities; establishing liaisons with business, industry, and other stakeholders; 
sponsoring public awareness and equity-related events; and leveraging funds. Program 
components that had stalled under central project leadership, such as Community Science, were 
also incorporated into the work of the Regional Coalitions. 

Similarly, moving from the centralized summer institutes to a regional format built around 
shorter, targeted conferences and on-site assistance from teacher leaders was seen as key to the 
Vermont SSI’s impact on schools. Combined with this change, a shift from working generally 
on inquiry and standards-based teaching to a specific effort to translate the mathematics and 
science Frameworks into local curriculum provided the “hook” for a sustained presence in 
participating schools. 

While greater regionalization gave the SSIs more of a “grassroots” flavor, which was particularly 
important in states with a history of local control, from the perspective of a number of SSIs, this 
trend was a mixed blessing. Regional structures could help SSIs provide more support to 
districts and schools, and/or to tailor their activities to the local context, but they also made it 
more difficult for the SSI to provide quality control. 

The Need to Improve Pre-Service Education 

Although the need to improve pre-service education was cited in many of the SSI proposals, 
there was considerably less activity in this arena than in in-service education (Zucker et al., 

1998), and a notable lack of success in operationalizing the reform vision when they did attempt 
it. The interventions had a very tentative and exploratory flavor, such as conferences/forums/ 
working groups to discuss teacher preparation issues; mini-grants to colleges and universities to 
gather a group of faculty involved in mathematics, science, and teacher preparation to assess their 
institution’s current programs and to initiate change; or pilot projects to redesign a particular 
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content or methodology course for pre-service teachers. Several SSIs took the approach of 
working to develop a shared vision for the preparation of mathematics and science teachers, and 
creating guidelines or “voluntary standards” to help interested colleges and universities 
implement this vision. 

In interviews, SSI Pis and key observers noted some of the barriers encountered in trying to 
make inroads into pre-service education. Said one observer: 

We ’re talking about different cultures. The Department of Natural Science is a culture, 
the Department of Education is a culture, teachers are another culture, and now the 
education faculty is another culture. The SSI was on the right track, but their head and 
their effort were looking somewhere else — the schools and the teachers. They thought 
teacher preparation reform would be easier to do than it was. But they underestimated 
the difficulties. 

One SSI that did an in-depth analysis of the teacher education programs in the state and attacked 
the problem of reforming pre- service education directly characterized the experience as 
“painful.” 

When you went down and you looked at what is the quality of the [pre-service] 
instructional program based upon what we ’re asking K-12 teachers to do with children, 
based upon what the children must demonstrate as active and constructive knowledge, 
there was a mismatch. And so we sat down and spent a year developing a set of 
recommendations. ... [When we presented them to college and university faculty, they] 
sat there and looked at us with blank stares and said, “Why in the hell should we 
change? I mean we are nirvana, why should we change?” And we scud, “ Because the 
product you ’re putting out isn 7 up to what we require for children in the K-12 
community. And then of course that began battles [because this was happening [ during 
the math wars, so you had the math faculty in the schools of arts and science saying, 
“What you 're doing with standards and what you ’re doing with the mastery test and 
what the NCTM is saying is all wrong anyway. ” So in the context of those battles we 
were trying to create this sense of unity and cohesion, and it wasn 7 fun. 

Persistence paid off, however. In Phase II of this initiative, the PI reported success in 
operationalizing at least a part of the pre-service education reform through an induction- 
mentoring process, where each new teacher develops a portfolio of teacher activities and student 
work by the end of his/her second year. 

We have been able to get more and more higher education faculty involved in that 
process. So we had been pushing for the last seven years, ... [that] it could be a 
wonderful quality feedback system to the institutions of higher education, because here 
we have new professionals that are demonstrating their skills working with children, in 
the context of content and pedagogy and management skills and cdl the other issues that 
take place during the art of teaching, and feed back to the institutions how well they were 
preparing their [teachers] because we could show them. There was a reluctance, if not 
reticence, of higher ed to pay attention to it. They smiled, took it. Now, in the year 2002 
it is becoming institutionalized, where we sit down and we have deep discussions about 
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the quality of each year’s cohort. . ..From where higher education has been, we ’ve made 
some movement, but it’s certainly been hard. 

Refining the Interventions and Providing Evidence of Impact 

Once the reform interventions were underway, SSIs had an opportunity to refine them, and to 
discontinue those that were not working well. The following sections describe the extent to 
which the SSIs made these kinds of mid-course corrections, and the problems they encountered 
as they tried to demonstrate the effectiveness of their reforms. 

Fine-Tuning the Interventions 

The SSIs put a great deal of effort into getting the reform efforts up and running. Some included 
pilot studies as part of their plan, in order to identify problematic implementation issues before 
they took pieces of the initiative to a larger scale. This approach was particularly valuable in 
SSIs that were not intending to provide direct services to large numbers of teachers themselves, 
but rather were planning to work through existing professional development networks in the 
state. Explained one of the Pis: 

[We believed that ] if you teach the standards and you teach them well, as they were 
designed, you will see high levels of performance on the test. That influenced us to work 
in some districts to make sure it happens, because just working through intermediaries 
we've got quality control issues. And so we’ve tried to work directly in some districts... 
to learn what it takes to make that happen. 

Some SSIs had state and/or national experts review the products they produced, both policy 
documents and materials intended for broad dissemination, and used feedback to improve the 
products. SSIs also worked to ensure that the services they provided had the maximum 
probability for success, capitalizing on what they had learned in their pilot studies and in earlier 
intervention efforts. For example, the Arkansas SSI required prospective professional 
development providers to attend two weeks of training to help ensure that those who might rely 
on more didactic modes in their own teaching would teach the SSI courses as intended. Despite 
some initial grumbling, all agreed to participate in the training. 

Similarly, the Puerto Rico SSI prepared the ground for the interventions to take root before they 
began to provide services. Said the PI: 

The whole school strategy in Phase I was critical. If I had to say one thing that we have 
done that has helped to move this thing forward, it was coming up with the idea of having 
cdl of the science and mathematics teachers and the principal [in a school] agree that 
they wanted to do this. It forces the situation of thinking as a group and working as a 
team. 

SSI staff sometimes monitored the intervention activities, and other times project evaluators did 
so, providing feedback on the quality of implementation of the interventions to enable project 
leaders to make informed mid-course corrections. Some SSIs created committees to assist with 
monitoring the activities of the initiative; these groups conducted interviews, surveys, and focus 
groups with key players, and reported back to meetings of larger committees. The Michigan SSI 
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included evaluators on its management team from its onset and the Nebraska SSI added 
evaluators to its management team in order to provide feedback on a regular basis to improve 
strategic planning. In Maine, an entire component of the initiative, Systemic Planning and 
Evaluation, was created to facilitate evaluative feedback, self-assessment, and decision-making. 
The leader of this component said: 

The self evaluation, it’s easy to say, but your ego starts to get involved, unless you can 
really do the self evaluation it’s hard to understand how you make progress. So creating 
that internal commitment to evaluating how we are doing I think has got to happen. 

Redirecting Efforts that Appeared Problematic 

Comparisons between the initial proposals and final reports indicate that some SSIs rolled out 
their initiatives essentially as planned, spending their time and resources pretty much as they had 
proposed. Still, it was common for various parts of the plans to fall by the wayside simply 
because there was not enough time, and/or not enough money to do it all. 

The most strategic SSIs were able to redirect their efforts, sometimes considerably, when they 
saw emerging opportunities. For example, the Texas SSI changed its strategy from a focus on 
establishing regional centers to a greater emphasis on curriculum guidance mechanisms in order 
to leverage the system to a greater extent. Explained the PI: 

It became clear to me that we were in an unusual position of being able to use the state ’s 
policy framework to support the systemic reform efforts in math and science. And 
therefore it would be worth carving out a bigger domain of policy work, so we had to 
[create a] strategic plan for this. ...The first thing was [to] manage the standards, and 
get at least one of the symbolic course areas, like Algebra, unanimously adopted by a 
contentious board. ... So we got it and it was adopted universally, by everyone from the 
Eagle Forum to the NCTM folks. 

Other SSIs demonstrated their strategic nature by changing approaches when they saw that what 
they were doing was not working the way they intended. For example, the Connecticut SSI had 
been providing grants to enable districts to implement reform tailored to their particular contexts, 
but decided to switch gears when the resulting efforts did not measure up to the SSI standards. 
The PI explained the reasoning behind the change: 

These school districts that we’d identified would submit plans for what they were going 
to do based on our goals and we would give them grants of money to have them go off 
and do the work, and then we would sit back and analyze the results of that work, and 
what we found out was nothing was changing. It was benign ignorance. They didn 7 
steal; they didn 7 cheat; they just didn 7 know what to do. They kept doing what they ’d 
been doing, and it was more of the same old stuff. 

In the third year [we began ] having serious talks with the Pis who were the state ’s math 
and science consultants, about the fact that giving this money to school districts was not 
a reasonable, reasoned, or sensible strategy. And that we had to quit giving them money. 
There was a great deal of concern at the leadership level, that if we quit giving these 
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school districts money that they would quit working with us, that they would say, “If 
we ’re not getting any money from you why should we work with you, because you ’re 
forcing us to do stuff that we don ’t want to do ? ” What we said, what the Academy said 
to the State Department of Education was, “So what? What we’re doing now isn 't 
working; it’s just adding money to the same old thing, and that’s not what systems 
improvement is all about. ” 

... We were all learning new things. I mean this was a real mindset change, and a way- 
of -doing-business change, and so after the third and beginning of the fourth year we told 
the districts we are no longer going to give you money, but what we will do is we will 
provide you technical assistance to do different things, and some of that money, if you 
need to find substitutes and that kind of thing, we may help you there, but what we’re 
really going to do is we’re going to provide you technical assistance. 

Other SSIs realized that particular components of the initiative were not working well, but could 
not see a way to improve the situation without alienating key partner groups and thereby putting 
the whole enterprise at risk. A PI of one such initiative noted that in retrospect, he realized that 
their project design had a major flaw from the beginning: 

[We made a] critical mistake. I’ll tell you this; I looked back at the negotiation as we 
started to pull the team together to write the proposal. I wasn ’t involved in it, the first 
year that we wrote a proposal that did not get funded. I wasn ’t the leader then. We went 
to people and we said “You ’re a critical player, what do you want to do ? ” Huge 
mistake, HUGE mistake. 

. . . You ’d see which partners were able to carry out what they said they were able to do. 
That was really one of the disappointing things. That’s where the mistake was. We 
would parcel out something to a particular institution or a particular area of the state 
and they were just unable to do it. [But it was seen as an] absolute entitlement, and 
that’s what killed us. [Other efforts] had very little to do with the changes that we 
wanted to make in science and math, but they basiccdly said that if you cut us out we ’re 
going directly to NSF, we’re going to do this, we’re going to be in the newspapers. It 
was bad. 

Another key leader in this SSI expressed a similar view, noting that people may have said, “I can 
do this” but it may not have been what was really needed. When asked what they would do 
differently in hindsight so that all of the key players would have an incentive not only to 
participate but also to modify their plan to fit within a larger whole, this individual talked about 
the need for more negotiation about roles. 

I think it’s valuable to have people working on things that they have some history with 
because they ’ve got people that bring expertise to it, but I think what I would do 
differently with that is really negotiating [to make sure that there is] value added. What 
is this going to do that hasn ’t [already] occurred? 
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Similarly, the PI emphasized the need to establish the broad goals first, and distribute money 
based on a group’s ability to accomplish those goals. 

Well, rather than dole out pieces to institutions and making those pieces somewhat unique 
to that institution, / think we ’d talk about broad goals that we need to accomplish and 
develop a statewide plan to accomplish those. Then based on success you award money 
around various parts of the state. So not based on an entitlement, but based on 
accomplishmen t. 

Linking the finances of the SSI to accomplishing vital aspects of an overall plan could allow not 
only for greater quality control, but also for smoother transitions if the plan needed altering in 
mid-stream. These advantages, however, would have to be considered against the possibility of 
losing players in the reform due to weaker financial incentives and a perceived lack of trust 
among partners. 

Providing Evidence that the Reform Interventions Lead to Improvements in Teaching and 
Learning 

Having a vision based on national standards, using “research-based” interventions, putting in 
quality control procedures for products and implementation, and even using monitoring results to 
refine/redirect efforts, still does not guarantee that the interventions will in fact lead to the desired 
outcomes. Once the reforms were underway, the challenge for the SSIs was not simply to make 
sure that they could be implemented as planned, but also to make sure that they were effective. 
There was a clear trade-off, however; taking the time to fine-tune the interventions to ensure their 
efficacy meant they could not reach as many teachers as quickly as they had hoped. Most often, 
the SSIs moved ahead with as wide an implementation as they could handle, trusting that the 
interventions would indeed produce the desired outcomes. 

It is interesting to note that quite a few of the SSIs used a “model sites” approach to the reforms, 
planning to get the reforms underway in a manageable number of places before scaling up, but 
did not go as far as they needed to go in showing that their models actually produced the intended 
outcomes. Often, the model sites were used to show that the reforms could be implemented, but 
they were not subjected to systematic evaluation, leaving the SSIs unable to show that the models 
in fact had the intended impact. 

SSIs soon learned that there were marked differences between the “soft” data that are sufficient 
to alert project staff to the need for making modifications to a program component, and the 
carefully-designed studies necessary to convince external audiences of the value of the initiative. 
But designing and implementing appropriate studies proved daunting to many of the SSIs. 
Sometimes there were concerns about the nature of the instrumentation to be used. On the one 
hand, the SSIs wanted to use student assessment measures that would both model the assessment 
strategies they were advocating for classroom use and be sensitive to the changes in curriculum 
and instruction they were trying to promote. On the other hand, they recognized the need to use 
assessments that would be credible to key stakeholders. One PI noted that in his state, the key 
consideration was using externally-developed tests in order to avoid concerns about “teaching to 
the test.” 
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And the ones the businessmen had credibility with, the people we needed to keep the thing 
moving, were tests ... that were not created in [the state]. And that was the only thing 
that the business community really cared about. They cared about AP; they cared about 
Iowa, Metropolitan, anything we could scrounge, and NAEP. 

SSIs sometimes found that their state data systems could not, or would not, provide the data they 
needed to assess the effectiveness of their interventions. For example, the SSI in one state tried 
to get data from the state department of education, but hit a brick wall, with agency staff claiming 
that any release of data, including summaries without student-identifying information, would 
violate student privacy laws. Even getting the governor to intervene did not succeed in getting 
them the information they needed to assess the impact of the SSI. They were forced to rely on 
the data districts were willing to track for them, which resulted in far less than a complete set of 
data. 

Other SSIs faced similar problems. In one, the statewide proficiency test was the metric many 
people in the community cared about, so the SSI wanted to use it in assessing the impact of their 
interventions. However, the state was unwilling to release results for individual classes, so it was 
not possible to compare the results of students whose teachers had and had not participated in the 
SSI professional development. In another SSI, the lack of a statewide assessment system and 
widely varying local assessment practices meant that the SSI “never had reliable [student 
achievement] data and struggled with it all the way through,” thus hindering any efforts to 
demonstrate advances in student performance as a result of the initiative’s activities. 

Even when there was agreement on the instruments to be used, and the student assessment data 
were available, SSIs found it difficult to design credible studies. One of the problems was the 
lack of convincing data on changes in classroom practice; without evidence that teachers who 
participated in SSI interventions had in fact changed their practice, it was difficult to attribute any 
gains in student achievement to the SSIs. One PI summed up the issue well, as follows: 

‘Assessment, attribution and accountability, ” that was our mantra, although it was not 
that clear initially. Intuitively, I knew it [was important ] from the beginning, but the 
articulation came out in the second phase. The weakest is the attribution. In social 
systems, the caused relationships are not linear, not simple. The best you can hope for is 
the quasi-caused relationship theit is persuasive. And the emphasis is on persuasion, not 
on proving. There are too many factors. 

In summary, very few of the SSIs were able to get the assessment piece “right.” In many cases 
they did not have the necessary data to work with, and either did not feel the urgency or did not 
have the capacity to fill the void. Not only did they lack the existence proofs that might have led 
to greater support within the state, but they were unprepared when NSF asked for evidence of the 
effectiveness of their initiatives. 

Increasing Capacity within the State to Scale-Up the Reform Efforts 

SSIs were far more likely to attend to getting their reform interventions underway than they were 
to think about how they would eventually be able to scale-up their efforts to affect the 
mathematics and science education of all students in the state. Although there were discussions 
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in proposal documents about building models of excellence and disseminating them widely in the 
state, and leveraging resources to enable them to expand their efforts, the scale-up strategies were 
not described in any detail; for many of the SSIs, the implicit strategy seemed to be to get it 
going first, and worry about how to scale it up later. 

The Puerto Rico SSI was one of the few which made its scale-up strategy explicit from the 
beginning, laying out a plan to reach a sizable proportion of the schools in the Commonwealth. 
The plan was to build capacity and credibility through piloting, evaluating, revising, and 
expanding strategies. Leaders were deliberate in their choice of reform strategies — starting 
small, using the school as the unit of change, and using regional centers as “test beds.” Each step 
would lay the groundwork for the next, with reform leaders moving as slowly and systemically as 
time and resources would allow, building capacity of core team members, major partners, and 
school staff along the way to ensure readiness for change. More typically, SSIs had some notion 
initially of how they might scale-up, and devised other strategies as the initiatives unfolded. 

Developing Model Sites that Others Could Replicate 

One of the key strategies for scale-up was to use model sites as examples that others in the state 
would replicate. For example, in the Georgia SSI, professional development schools were 
expected to become “models of innovation in mathematics and science”; after the third year of 
the SSI, educators from throughout the state were to visit these schools to experience the reforms 
on location. Similarly, in Delaware, Maine, Michigan, New Jersey, Puerto Rico, South Carolina, 
and other states, the intent was to use the SSI to enable teams of teachers and administrators in 
schools/districts to undertake changes in their science and mathematics programs that could be 
tested, refined, demonstrated, and disseminated widely to provide examples for others to 
consider. In some cases, the SSI intentionally planned to develop several different models, in the 
hope that other communities would be able to adapt aspects of one or more models to address 
mathematics and science education reform in their particular context. 

SSIs encountered major problems with the model sites strategy. First, it often took far more time 
and resources than they had anticipated to get the “models” to the point where they were 
comfortable using them as exemplars. Second, the mechanisms for accomplishing the scaling up 
were far from clear, leaving a lot to chance. Even if other schools/districts wanted to replicate 
the models, or use them as inspirations to develop their own models, it would have been 
extremely difficult for them to do so without the levels of expertise and resources that had been 
made available to the model sites. 

Increasing Capacity to Deliver Services 

Developing new professional development providers and educating existing professional 
development providers was a focal point of strategies and activities for many SSIs. A few 
initiatives began with the idea that such leadership development was a necessary piece of the 
reform puzzle, but many more came to realize that the SSI leadership by itself could not deliver 
the breadth and depth of change embodied in their reform vision. In order to reach broadly 
across a state and influence deeply the teaching and learning of science and mathematics 
simultaneously, the SSIs had to find ways to leverage the efforts of the groups that were already 
providing professional development in the state or could be prepared to do so, and to make 
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savvier consumers of the local education decision-makers who would choose professional 
development services for their schools and districts. 

SSIs attempted two basic strategies for developing leadership among professional development 
providers. In states with limited capacity for mathematics and science professional development 
prior to the SSI, the initiatives devoted considerable resources to developing professional 
development providers. In states with a number of already active professional development 
providers, SSIs often tried to work with existing providers. Some states attempted a hybrid of 
these approaches. In all cases, though, sustaining leadership for high quality professional 
development beyond the funding period of the SSI was a critical concern. 

Quite a few of the SSIs worked to develop teacher leaders as a means of scaling up their 
initiatives, engaging these individuals in fairly intensive professional development and then 
arranging for them to provide workshops to other teachers, in some cases in teams that also 
included scientists or mathematicians as content experts. SSI leaders and evaluators who 
observed workshops and institutes provided by teacher leaders typically reported considerable 
variation in quality, concluding that the level of professional development that had been provided 
to the teacher leaders was not sufficient for many of them in turn to be able to provide quality 
professional development to their peers, or to otherwise serve as leaders in their districts and 
schools. A leader in the Vermont SSI described how they were able to change their design to 
provide additional support to their teacher leaders. 

The original model was based on a faith in the ability of professional development to 
enable teachers to go back and be leaders in their schools. What [ the SSI] found over 
the first three or four years was that that was important, but it wasn 7 sufficient. The 
thought was that if we can bring teachers in to work in a leadership capacity in [the SSI] 
for a couple of years, get intensive experience in content, in pedagogy, in leadership, 
then go back to their schools, they’ll have the leadership skills to move things forward. 

A similar strategy was used in the Delaware SSI, where the initiative worked to create a cadre of 
teachers and community members — mostly other school-based professionals — who would be 
available as “culture -change agents” to provide technical assistance to schools. Of course, the 
strategy of developing teacher leaders who could then provide direct services to a large 
proportion of teachers in the state was far less feasible for the larger states, some of whom had as 
many teachers as Vermont or Delaware had students. 

Many states have infrastructures charged with providing technical assistance to schools and 
districts and professional development to teachers on a regional basis, variously called Education 
Service Cooperatives, Mathematics and Science Centers, or the like. One strategy for scaling up 
was for the SSI to join forces with these groups in order to expand the number of teachers 
reached. In fact, in a few cases, the SSI intended to provide very limited direct services itself, 
reasoning that they could have a greater impact by leveraging the work of these existing groups. 

SSIs experienced difficulty with this scale-up strategy, in some cases reaching the conclusion that 
the existing professional development activities were not consistent with their vision, and that 
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they could not rely on these mechanisms for providing the quality services needed to help 
teachers improve their practice. Said one leader: 

Most of what we found was embedded in the traditional model of the one-shot workshop, 
expert lecture, the university model. The need was clear to work with these intermediate 
organizations and individuals who rubbed up against school districts, who had 
professional development roles and responsibilities, and to build their capacity and 
develop strong communication across various players. 

The Michigan SSI is one that put its professional development emphasis on existing providers. 
One leader described this focus as an effort “to build their capacity and develop strong 
communication across various players.” The SSI helped arrange a two-day conference for 
participants to learn about newly emerging best practices (e.g., ongoing, job-embedded 
professional development), and to initiate the development of statewide professional 
development guidelines. Subsequent meetings for state leaders in mathematics, science, and 
professional development identified “action steps” needed to overhaul the professional 
development system. State affiliates of NCTM and NSTA, as well as Michigan Department of 
Education curriculum development staff and Eisenhower coordinators, were all involved in these 
efforts, attending quarterly meetings with SSI co-directors to plan and refine strategies for 
building learning communities through the regional Mathematics and Science Centers. 

At the end of the Michigan SSI, some leaders saw definite progress in this area: 

The Mathematics and Science Centers are building the capacity of teachers to examine 
student work, and teachers are coming together on a monthly basis. That would have 
been unheard of in 1992. (SSI Leader) 

We didn 7 get there by the end of the SSI. It just took a lot longer than we thought. But 
we made progress and [promoted] a lot of conversations that would not have happened 
without us. ... [Providers are] facilitating teacher conversation — helping teachers take 
the learning back to their school so they can continue the conversation. (SSI Component 
Coordinator) 

One observer, however, noted limited success in terms of systems change: 

There ’s no question that work with leadership resulted in an impact on the 
organizations — MCTM, MSTA, the Mathematics and Science Centers — those 
organizations ’ capacity was increased. The downside was that we didn 7 influence the 
professional development system in Michigan or didn 7 bring much coherence to it. (SSI 
Evaluator) 

In Nebraska, the SSI addressed the Education Service Units (ESUs) responsible for a great deal 
of professional development across the state by pulling them together with districts, higher 
education, and other resources in regional coalitions. Developing leaders among professional 
development providers was a key task. The effect on leadership among professional 
development providers in the state was highlighted as a notable success by one SSI leader, “The 
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professional development delivery system is much stronger now and much less variable. The 
Regional Coalitions helped ESUs to work together — weak ones and strong ones — on a regional 
basis,” resulting in a “more uniform level of excellence” in the professional development 
delivered by the ESUs today. However, the strategy did not work equally well throughout the 
state, as one leader acknowledged by saying, “Every Coalition needed someone with vision, and 
only about half of them had it.” 

Reducing Incentives for Participation to Provide More Resources for Scale-up 
In interviews, SSI leaders talked about the importance of districts contributing some of their own 
resources to the reform effort so that they would have a stake in its success. For example, in 
Nebraska, as part of the agreement to participate in the SSI, districts/consortia were to provide 
lead teachers with eight release days, and allocate Eisenhower funds for lead teacher preparation 
and the workshops they would conduct for their colleagues. In asking districts to share financial 
responsibility for staff development, reform leaders had to overcome local perceptions that “the 
university was taking money from us again.” But according to one PI, “We held the line, with 
the idea that resources are limited, and we’ve got to give [resources] to people who understand 
[the reform vision], and value it, and are willing to pay for it.” In return, districts received SSI 
funds to provide support to teacher leaders to work with their peers. Similarly in other SSIs, 
schools receiving services were expected to come up with the resources to support the services, 
and to provide the instructional materials to enable teachers to implement new practices 
afterward. 

As their work became better known, some of the SSIs leveraged their resources by decreasing 
monetary incentives for participation. For example, a number of initiatives that had provided 
mini-grants to districts or universities to “try new things” realized they no longer needed to do so. 
As a leader in the Texas SSI explained, “We were in a different stage of our work. We didn’t 
have to be handing out money at that point. People would try things because the quality of our 
work by that time was speaking for itself.” 

The strategy of providing incentives and then reducing them proved to be tricky for some SSIs, 
though. Two Pis recalled the problems their initially large incentives caused: 

We started off paying too much and it ... haunt [ed] us; that really lived to be true. It’s 
taken us again four or five years to break that trap of high paying stipends. . . . Yes, for 
us the weaning took a while for all of the districts. We had some willing to pay us very 
early, but then they were stuck with the high stipends. 

Everybody that came to the [professional development ] went away with a big, big bunch 
of manipulative s. As it turns out the teachers were just delighted to get it, it didn ’t matter 
where it came from. And some administrators saw that as a very negative fix, because 
they didn ’t want have to put up for the material — put up the money. And that, over time 
may have precluded some of our smaller districts from participating. 



Horizon Research, Inc. 



37 



March 2003 




Providing Evidence that Quality and Impact Are Maintained During Scale-Up 

A strategic statewide systemic reform plan, as we have defined it, must have not only 
mechanisms for scaling up the reforms in order to affect teaching and learning for all students, 
but also a means to ensure that they are able to maintain quality while scaling up. 

SSI staff and evaluators were fairly likely to monitor the quality of implementation of their 
interventions as they scaled up. But credible evidence of quality of services does not carry the 
same weight as credible evidence of impact on teaching, and especially, learning. Given the 
difficulty SSIs had in assessing the impact of their interventions on students in the first place, it is 
not surprising that SSIs were rarely able to compare the impact of the various phases of their 
interventions. Particularly in states that adopted a model school strategy, assessing impacts on 
student achievement beyond the model sites proved challenging. Although the SSI leaders might 
know which other schools or teachers had been “reached” through the model sites, tracking the 
nature and extent of intervention for these schools and teachers was not simple. Linking the 
interventions to changes in student outcomes was even more problematic. 

A notable exception was the Puerto Rico SSI, which put considerable resources into measuring 
student outcomes in “second-generation” schools, those where the majority of the intervention 
was carried out not through direct services from the SSI, but rather through select “first 
generation” schools that had been developed into centers for disseminating the reform in their 
regions. The Puerto Rico SSI PI described the strategic decision to seek evidence of impact as 
the initiative scaled-up: 

The resources are never commensurate with the challenge. They are catalytic. You 
detect the pressure points and use them sparingly at the right points. We ’ve been good at 
that. For example, it takes $300,000-$400,000 for the pre/post tests, but that is money 
well spent. 

The costly assessment system, which was developed, implemented, and analyzed almost entirely 
with SSI resources, paid off. The gains in student performance in the second generation schools 
were commensurate to the gains achieved in the first generation schools. The SSI could provide 
evidence not only that its scale-up strategy was a viable way to provide services to an increasing 
number of schools, but also that positive impacts on student learning were maintained. 



Political Strategies for Systemic Reform 



“Anyone who doesn’t understand that reform is a political game doesn’t know 
anything about reform.” (SSI Principal Investigator) 



Through its political strategy each SSI specified the activities that, if accomplished, would align 
the formal policies and capture professional, political, and public support to pursue its vision of 
teaching and learning. A strong political strategy would address challenges for building the 
necessary will for reform. 
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First, the initiatives had to consider the policy environments in which they operated. This 
required: understanding how policies guided actions throughout the system; assessing policy 
alignment with the reform vision for teaching and learning mathematics and science; and gauging 
policy incentives for teachers, schools, and districts to pursue the vision. Where guidance was 
unclear, alignment was missing, or incentives were weak, the SSI had to establish means to 
influence the policies in ways consistent with the reform vision. 

Second, the SSIs had to stimulate broad understanding and support for the reform vision, or at 
least find ways to minimize opposition. Typically, there were many groups in a state that had 
strong stakes in mathematics and science education. The leadership of the SSIs often represented 
only some of those groups, and even then the “rank and file” of the represented groups did not 
necessarily share the leaders’ commitment to the reform vision. The changes in mathematics and 
science teaching and learning that the initiative’s reform vision advocated had to be at least 
understood, or better yet, favored, by important stakeholding groups in order for the reform to 
succeed. 

Third, support from some stakeholders was particularly important. Decision-makers in districts 
and schools needed to grasp how the mathematics and science education reform interventions 
addressed key needs in their locales as a part of a coherent plan for reform, and needed to commit 
their schools/districts to providing the resources and support required to make the interventions 
work. The SSI needed to make a strong case that the interventions it could offer or broker would 
address these key needs in order to foster the will of education leaders across the state to adopt 
and address the reform vision. Service providers needed to understand the quality and intensity 
of intervention the reform required; the SSI needed to make a strong case that interventions of 
this quality and intensity constituted the appropriate services to provide. 

Our criteria for a sound political strategy, then, are that the strategy: 

• Facilitates development of formal policies that provide guidance and incentives for the 
reform vision; 

• Cultivates broad understanding of and support for the reform vision; and 

• Increases school and district leadership commitment to reform. 

The classic notion of systemic reform in education (Smith & O’Day, 1991) placed the upgrading 
and alignment of state policies with a reform vision of excellent and equitable education at the 
forefront. Decentralized decision-making was favored so long as appropriate incentives for 
meeting the requirements of policy were established. Our conceptualization of systemic reform 
in action maintains the centrality of policy improvement and alignment, but also recognizes the 
need to foster professional, political, and public will in more informal ways, and places all of this 
work alongside the development of leadership capacities and specific interventions for 
improvement in teaching and learning as key tasks of systemic reform. Our reasoning suggests 
that work on formal policies and the cultivation of support and leadership provide an 
environment in which systemic reform can succeed, but by no means provide a guarantee that 
systemic reform will succeed. This chapter examines the SSIs’ work in light of our definition of 
a sound political strategy, highlighting the reasoning behind what the SSI leaders chose to do in 
this area and how successful the SSIs were in meeting common challenges for reform. 
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Facilitating Development of Formal Policies that Provide Guidance and Incentives for 
Reform 

As noted earlier, many SSIs chose to provide intensive direct services to a large number of 
teachers in the state. An alternative strategy, or a companion strategy to providing direct service 
in order to scale-up reform efforts, would be to modify the documents that provide guidance to 
teachers in their decisions about what mathematics and science to teach, and how. A number of 
the SSIs took this approach, working on the development of state curriculum frameworks and 
assessment instruments to reinforce and clarify high standards and help drive quality science and 
mathematics instruction. 

Few of the SSI leaders had backgrounds in education policy, but many recognized, or came to 
recognize, how central formal policies were in guiding and providing incentives for the kind of 
mathematics and science that was taught and learned in schools and classrooms. At the time the 
SSIs were being planned and initially implemented, a national trend toward more centralized and 
more powerful state policies was just beginning. This trend has continued for more than a 
decade. Even in traditionally local-control states, there was significant movement toward 
statewide policies providing guidance and incentives for teaching and learning. These shifts in 
the centralization of policy were being met with considerable resistance in some states, but the 
trend continued nonetheless. The SSI in Connecticut began in a system well along on this trend. 
SSIs in states such as Vermont, Nebraska, and Maine entered systems just moving toward more 
centralized policy guidance. 

At the same time, a national trend toward decentralization of decision-making to respond to 
centralized policy mandates was underway. In states with strong centralized policy guidance 
already in place, local agencies were being given greater responsibility for making their own 
decisions about how to meet requirements, which were often being enforced with increasingly 
strong accountability measures. The Texas, Michigan, and South Carolina SSIs, for instance, 
began in systems where already influential centralized policies were being strengthened further, 
but were being coupled with measures to give greater autonomy to districts as the decision- 
makers with authority to respond to policies. 

The original SSI solicitation encouraged proposers to work on certain policies, in particular, 
“policies and programs related to the initial preparation and continuing education of teachers,” 
“essential curriculum content,” “learning goals,” “instruments and techniques employed for the 
assessment of student progress and achievement,” and “policies and procedures by which 
individuals and institutions are held accountable for science, mathematics, and engineering” 
(NSF, 1990). Moreover, beginning in 1995, NSF required SSIs to report their activities and 
progress annually in specific areas. These “Drivers of Systemic Reform” further focused the 
attention of the SSIs on the policy arena, in particular through: 

Driver 1: Implementation of a comprehensive, standards -based curriculum and/or 
instructional materials that are aligned with instruction and assessment 
available to every student served by the system and its partners; and 
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Driver 2: Development of a coherent, consistent set of policies that supports provisions 
of broad-based reform of mathematics and science at the K-12 level. 

Conducting work on policy was a way for the SSIs to provide widespread guidance in 
accordance with the reform vision, and in some cases, to create incentives for districts, schools, 
and teachers to participate in the reform. Ways to engage the SSI in policy work included: 
positioning the initiatives to contribute to policy development and refinement; establishing 
credibility as a player in policy development; and remaining alert to important opportunities to 
influence policies that affected mathematics and science education. In the ideal, the SSI would 
take all of these approaches, attempting to build a coherent policy system that supported the work 
of the SSI and provided incentives for districts, schools, and teachers to participate in and value 
the reform. 

Positioning the SSI to Engage in Work on Formal Policies 

Among the first decisions each group of SSI leaders had to make were where to house the SSI as 
an organization, and which other leaders in science and mathematics education in the state to 
include in developing and managing the initiative. These decisions often proved critical in 
determining the SSI’s ability to define a role for itself in reforming the state’s mathematics and 
science education policies. 

A number of SSIs were positioned so that the SSI would naturally engage in important policy 
work. In some cases, the SSI deliberately oriented itself to be a key player in policy 
development. In other cases the SSI developed a sense of the need to influence key policies over 
time, but found itself in a strong position to do so when the need became evident. 

An important means of positioning the SSI to engage in policy work in support of systemic 
reform was to begin with close organizational ties to agencies that developed or influenced 
policies. Another approach was to assure that individuals with influence over the crafting of 
policies were included as leaders in the SSI. For example, the Puerto Rico SSI was established 
as a partnership among three vital agencies in the education system of the Commonwealth — the 
Puerto Rico Department of Education, the governor’s General Council on Education, and the 
Resource Center for Science and Engineering at the University of Puerto Rico. The PI had an 
established reputation for quality work in education and a position on the Council of Advisors to 
the governor. 

The South Carolina SSI was similarly positioned to engage in policy work relevant to 
mathematics and science education. Closely tied to the South Carolina Department of Education 
through its leadership and organizational home, the SSI also enjoyed strong support from the 
State Superintendent. Working on policies in mathematics and science education was a natural 
thing for the SSI leaders to do and when the focal policies in the state shifted from curriculum 
alignment to school improvement planning, the SSI’s organizational position and high level 
leadership allowed it to retool its policy focus, while keeping mathematics and science central to 
policy decisions. 

In Michigan, the SSI was inaugurated within the state Department of Education, including key 
leaders in mathematics and science education in the initiative’s leadership. Because it began in a 
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system that was perceived to already have a fairly strong and supportive policy environment for 
systemic improvement, one observer said, “[The Michigan SSI] looked to see where there were 
gaps. For example, they chose not to focus on issues around standards because [that] was already 
in the works.” The SSI defined its role in policy work as that of a critical friend, reviewing 
existing policies and their implications for districts and schools and making recommendations to 
policy-makers in an effort to create a policy system with greater “power and coherence” for 
district and school improvement in mathematics and science, especially for underserved students. 
Michigan SSI leaders in positions to influence policies would use the reviews and 
recommendations to leverage changes in policy. 

Delaware’s SSI was physically and somewhat organizationally separate from the Department of 
Public Instruction, but its PI was a member of the Department’s cabinet. This arrangement 
allowed the SSI to have a voice in policy discussions and decisions. With its close organizational 
ties to the University of Delaware’s Research and Development Center and the Department, the 
SSI was in a position to serve as the Department’s implementer and tester of new policies being 
legislated under the state superintendent’s “New Directions for Education in Delaware” reform 
movement. Due to the positions and expertise of SSI leaders, the state Board of Education 
charged the SSI with a review of state policies and practices that help or hinder reform. 

A few SSIs set themselves apart from state agencies in their organizational homes, but were sure 
to include important agency leaders in the initiative’s leadership. For example, both Maine and 
Connecticut established new non-profit agencies to house the SSI, but both did so with very close 
ties to the state departments of education and the inclusion of key mathematics and science 
leaders as Pis. The Maine SSI was physically housed at the Maine Department of Education, 
despite its independence from the Department. The Connecticut SSI was both physically and 
organizationally separate, but was sponsored by important state agencies — the Department of 
Education, the Department of Higher Education, and the Department of Economic Development. 
Although these arrangements did not assure the full involvement of the SSI in policy work, they 
set the stage for the SSIs to stay abreast of opportunities and to establish themselves in policy 
work over time, strategies which are discussed below. 

The Connecticut SSI took another important early step in establishing its position to work on 
state policies in mathematics and science education. As part of the inauguration of the 
organization that housed the SSI (the Academy), the leaders obtained a charter from the state 
legislature, as one Co-PI from the Department of Education said, “The initial thing is they got the 
go-ahead from the legislature to be the advocate for math and science, so that was legislated.” 
Positioning the SSI in an independent organization allowed broad participation, but involving 
key people from the Department of Education remained critical, as a Co-PI noted, “I don’t think 
there’s anything the Academy has done ... in terms of product that they haven’t involved the 
consultants here at the Department of Education.” 

Some SSIs, despite their strong organizational positions to influence policy, did not initially see 
this as a role for the SSI. One leader in New Jersey said: 

Our policy work came in as we began to have a better understanding of the concept of 

systemic change. A lot of our work in policy was cdready going on - we were involved 
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with the math coalition, the state frameworks work, etc. But the concept that this should 
all be seen as part of the SSI was new to us. After we had done a lot of work on the SSI 
that message came through to us clearly. 1 remember returning from a meeting in 
Washington, and saying that we need to bring this cdl together under one roof. 

Because the New Jersey SSI was fiscally managed by the state’s Department of Education, when 
it began to integrate policy work into its overall strategy, it was able to have significant influence 
on the development of curriculum standards and frameworks aligned with national standards, 
assessments that included science and technology, and teacher licensing and recertification that 
recognized the importance of content and technology preparation. The SSI’s specific work in 
equity also influenced this array of policies. 

Where the SSIs were organized in agencies that had little direct influence on state education 
policies, particularly higher education institutions, they often felt a need to reposition themselves 
as the initiatives unfolded. The Nebraska SSI provides a good example. Originally established 
through the Mathematics Department of the University of Nebraska-Lincoln, the SSI was not 
particularly oriented to policy work in its early days. As one of the Pis recalled, “We had no 
notion that we could affect policy. It either dawned on us later or it was pushed on us. But it 
wasn’t clear to us at the beginning.” This orientation was reflective of the state’s minimal role in 
educational guidance at the initiation of the SSI in Nebraska, where local control was extreme. 
During the SSI, however, the state moved toward stronger policy, accountability, and technical 
assistance roles. The SSI located its offices off campus to be situated closer to the Department of 
Education, and restructured its leadership over time to include a broader array of key players in 
mathematics and science, including key leaders in the Department and other state agencies. 

Establishing Credibility to Work on Formal Policies 

For many SSIs, engaging in policy work depended on the initiative establishing itself as a 
credible contributor to policy development. In a sense, the notion of an existence proof played 
out in this arena just as it did in the SSI’s work with teachers, schools, and districts. 
Demonstrating the ability to produce high quality work in one policy area helped the SSIs be 
seen as a likely source of expertise and contribution for ongoing policy work in mathematics and 
science education more generally. 

Some SSIs sought out opportunities to work first on specific policy developments as a way to 
demonstrate their ability to serve the state in this capacity. For example, the Texas SSI was 
reborn in the state at a time when the existing curriculum standards were being overhauled. 

When the Commissioner of Education sought a group to manage the standards creation and 
review process, the PI of the SSI, with the backing of the governor, offered the services of the 
initiative. The choice of where to begin was a critical one, as the PI described: 

The first effort was the TEKS [Texas Essenticd Knowledge and Skills ]. ... And then we, 
after that, because we had a broad political base, the legislature, in the next session, 
asked us to take on cdl kinds of analysis related to the legislative interim charges. ... We 
wrote a series of four reports for the senate education and house education committees, 
about the policies, issues that influence curriculum advancement in the state. 



Horizon Research, Inc. 



43 



March 2003 




The Vermont SSI was at its inception a natural partner for the small and resource-poor 
Department of Education to accomplish policy work. The Commissioner of Education’s 
ambitious plan to increase the influence of centralized policy guidance in the traditionally 
locally-controlled state education system set the stage for the SSL As one leader described: 

[The SSI] made a conscious decision to be an active partner in development of the 
statewide frameworks. That was very important, because they became the foundation for 
other work in the state. And through that process VISMT was able to build some real 
credibility for its capacity in math and science. 

The resulting Vermont Framework of Standards and Learning Opportunities became exactly the 
foundation the SSI needed to later engage in critical work on the state’s portfolio assessment 
system and the adoption of the New Standards Reference Exam as a formative tool for decision- 
making. Later, leaders from the SSI were a part of the decision-making process when the 
assessments became a key part of the state’s emerging accountability system. 

Another strategy some SSIs used to establish their credibility was to have their work reviewed by 
locally and nationally respected groups. A favorable review of the initiative’s work in policy- 
relevant areas in the state could open doors for the SSI to serve the state in policy matters. For 
instance, the Delaware SSI put considerable resources into its service as an essential partner in 
the state’s development of both a curriculum framework and an assessment framework. The 
curriculum framework received favorable reviews from the American Federation of Teachers, 
and the assessment framework was reviewed positively by national experts in assessment. The 
SSI then played key roles in writing, piloting, and refining the mathematics and science portions 
of the state’s standards and developed a bank of assessment tasks and scoring rubrics that 
influenced the revision of state testing and assessment recommendations. The SSI also had 
influence in the state on teacher licensing and recertification policies, and standards for 
administrators. Moreover, the Elements of Effective Practice , a document on best practices in 
teaching, and an evaluation instrument developed to measure these practices, have become part 
of state recommendations to districts to shape instructional practice. 

Early on, the Maine SSI developed the Curriculum Framework for Mathematics and Science, 
which has guided efforts during and after the SSI funding. Developing the Framework, one 
leader said, “moved us to begin to gain a consensus around what are the important things for kids 
to know and be able to do in science and math.” The Maine SSI had state and national panels 
review the Framework. Favorable reviews lent early credibility and visibility to the SSI, and 
ultimately positioned the SSI as a contributor to the Maine Learning Results, which are the 
state’s guiding principles, content standards, and performance indicators; and to ongoing 
revisions of the Maine Education Assessment. 

At times in the SSIs, credible, high quality work had an indirect influence on policy. The Ohio 
SSI’s focus on professional development around intensive, sustained, content-rich courses in 
teaching through an inquiry-based approach to instruction is now reflected in the state’s teacher 
licensing. Since the SSI, the state has instituted a middle school teaching license with a heavy 
emphasis on content preparation. Also, a PI reported, all science and mathematics pre-service 
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teacher programs, including those for elementary teachers, had to be revised to include inquiry- 
based content courses. 

In Georgia, the SSI engaged in considerable work in pre- service education, developing the 
Principles of Educating Teachers (POET) as a guidance document for this work. Following the 
SSI’s work, the state universities’ Board of Regents adopted a set of principles for teacher 
education in all disciplines. The PI noted: 

When we started POET it was the only initiative that I know of that tried statewide to at 
least get higher ed people to communicate about what was going on in teacher 
education. . . . All we could do is get people together to talk and to improve then- 
pro grams and give them a little bit of money to do that, but ... the Board of Regents has 
that power. So that’s a real success right there. 

The Georgia SSI also produced other documents that had influence on new policies, despite a 
shift in state politics away from the SSI’s reform vision. The SSI’s Learning Framework 
influenced revisions of the state’s curriculum standards, and the SSI’s Diversity Framework set 
the stage for equity considerations to be included in policy discussions and formulations. As the 
Project Director described it, the Learning Framework might not be seen as a direct influence, 
but “it did inform the revision of the state curriculum. . . . There were people that had been 
involved in the [SSI’s] work. . . . They used the documents . . . and the thinking that went into 
them.” About the Equity Framework, the Project Director said, “It was the first time there were 
real conversations going on around equity issues in the state. That was not a planned thing. That 
evolved in the SSI. . . . There are remnants of that.” 

Remaining Alert to Opportunities to Influence Policies 

A key strategy for influencing formal policies was being attuned to opportunities to work on 
important policies. In some cases, opportunities fell into the laps of SSI leaders due to 
organizational positioning or involvement of key personnel in the state. Often, however, even 
with a track-record of credible contribution to policy development, the SSI had to anticipate 
policy developments, or even shape their own opportunities. 

Engaging in relevant work in advance of state policy decisions was an important way for an SSI 
to influence policy proactively. As already described, the Delaware SSI’s work on assessment 
and the Maine SSI’s development of a curriculum framework put those SSIs in the lead in their 
states for work on related formal policies. The Connecticut SSI offers other examples of this 
strategy. When the SSI began, Connecticut had statewide testing in mathematics, but not in 
science. The SSI, along with the Department of Education, helped to develop and disseminate 
performance tasks in science and developed a science assessment instrument for districts to 
examine performance in science. All of this work, along with specific lobbying efforts, resulted 
in the inclusion of science as a content area on the 10 th grade statewide exit exam. The 
Connecticut SSI also worked on developing portfolios for beginning teachers, which have had an 
important influence on the state’s teacher induction system. Further moves to use the induction 
teacher portfolios as a feedback system to teacher preparation programs, and to institute policies 
to support this system, are ongoing. 
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Other means of anticipating policy development and staying a step ahead of the system have 
been employed in some SSIs. The Puerto Rico SSI used an influential report from the governor’s 
Mathematics and Science Advisory Board to develop its original proposal for the initiative, and 
has relied on the areas identified in this report — teaching practice, professional development, 
equity in delivery of resources and services, and management — to define much of its policy 
work. For example, the SSI advocated for, and contributed to, the development of Professional 
Development Standards that now influence pre-service and in-service teacher education and 
licensing and recertification requirements. Also, the SSI’s Regional Professional Development 
Centers have become the legislated means for delivering resources and services in mathematics 
and science, and recently other content areas, and for introducing, sustaining, and articulating 
improvements in professional development and instructional materials. 

When state curriculum and assessment policies (and accompanying incentives) were aligned with 
their reform vision, SSIs were able to use them for leverage. Staying apprised of the priorities at 
the state department of education and making sure that SSI activities were structured to help local 
educators be successful in navigating state reform were key strategies for several SSIs. Rather 
than pushing standards-based mathematics and science as the “right thing to do,” these SSIs took 
a pragmatic approach — “let us help you do what you’ll have to do anyway.” 

The South Carolina SSI has attempted to keep itself attuned to key policy movements in the state, 
particularly emphasizing work in areas with the potential for broad impact. During its first five 
years, the SSI put considerable effort at the state level into developing the state’s new 
mathematics and science curriculum frameworks. Locally, districts and schools paid careful 
attention to the frameworks, knowing they would be held accountable for delivering instruction 
aligned with these new policy documents. The SSI used its regional HUBs to aid districts and 
schools in interpreting and responding to these documents. Similarly, with statewide science 
instructional materials adoption approaching, the SSI worked at the state level on guidelines that 
would facilitate adoption of the highest quality materials available. Concurrently, the SSI 
reached out to local leaders and teachers to attune them to characteristics of high quality 
materials that they should be considering when the adoptions were undertaken. Later, sensing a 
shift in policy focus, the SSI turned much of its attention in its second five years to newly 
required School Renewal Plans, participating in state-level development of the policy in regard to 
mathematics and science programs. Again at the local level, the SSI used its regional structures 
to aid schools and districts in responding to the new requirements, with an emphasis on 
upgrading science and mathematics programs as a part of school-improvement planning. 

The Vermont SSI employed a similar strategy. Having spent several years contributing to the 
state’s curriculum framework in mathematics, science, and technology, and further work on the 
state’s assessment systems, the SSI used the adoption of the frameworks and assessment as a new 
jumping off point for its other work. One leader elaborated: 

At [the SSI’s] first conference, the focus was “What is inquiry? What is good practice?” 
and we stayed on that through Phase I. By the time we had the standards and statewide 
tests in place, the questions were changing — “How do you implement standards-based 
teaching?” We were beyond the awareness stage; “How do you get the systems in place 
in this new, standards-based environment?” 
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Similarly, the SSI used the passage of the Equal Educational Opportunity Act as an opening to 
further its work. This policy obligated schools to engage in data-based review and planning of 
finances, quality, and equity of instructional services. The SSI moved quickly to position itself, 
with the approval of the Department of Education, as a resource for districts to respond to the 
new policy, keeping a focus on high quality mathematics, science, and technology as a priority. 
The SSI’s orientation to new requirements has helped the initiative reach a considerable number 
of districts in the state. 

Cultivating Broad Understanding of and Support for the Reform Vision 

A vital consideration for the SSIs was finding ways to garner broad-based support in the state and 
to avoid destructive opposition. To accomplish these goals, SSIs had to determine who the 
important players in the mathematics and science education community were, who the important 
power brokers in the state were, and who in the broader community it was important to include. 
SSIs developed a host of means to engage and communicate with these individuals and groups. 

The SSIs pursued three main audiences in their efforts to cultivate broad understanding and 
support for the reform vision. Garnering support from highly placed decision makers was 
pursued as a means to assure that the SSI would be consulted on important decisions, and would 
have long-term financial backing to continue its work. Additionally, SSIs saw it as necessary to 
have many people, representing different sectors, regions, and localities of the state, aware of and 
vested in the reform vision in order to bolster the SSI as a legitimate force in statewide reform. 
Finally, parents and communities were targeted for outreach as special points of leverage for 
local support for changes in schools and classrooms. 

In support of these objectives, the SSIs pursued strategies that they considered well suited to then- 
context. At the same time, some of these strategies, and the lessons learned in their deployment, 
appear to generalize across contexts and may inform large-scale reform more broadly. First, 
establishing the SSI as neutral “turf’ politically was an important consideration in many states. 
Second, measuring, publicizing, and using results of quality and impact operated as a powerful 
way to gain support from key stakeholders in several SSIs. Third, facilitating the work of other 
entities in the state, and publicizing their successes, helped garner support for the SSI efforts. 
Finally, establishing the SSI as something that adds enough value to the mathematics and science 
education system to be seen as worth the investment of state and/or local funds has been an 
important strategy that some initiatives have employed as their efforts have matured. Many of 
these strategies are balancing acts for the initiatives. For example, being seen as a substantial 
force that delivers results can support the notion of adding value to the state mathematics and 
science education system, but can collide with the needs to establish neutral turf and to share 
credit for successes. Conversely, appearing as completely neutral and sharing all credit could 
allow the initiative to be seen as contributing little to the state education system. 

Involving Key Stakeholders at the State Level 

Many SSIs tried to identify and to include the state’s key “power brokers,” particularly policy- 
makers, in the reform. The SSI solicitation supported this approach to a degree, since each 
state’s governor and often other top policy-makers wrote letters of support for the proposals, but 
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many SSIs went further in their strategies to cultivate support for the vision among key power- 
brokers. 

The SSIs often involved the Chief State School Officer, college and university presidents, 
legislators, and representatives of the business community in their efforts. In interviews, several 
Pis talked about regular meetings with the Governor’s education advisor or with staff to key 
legislators, to keep them apprised of SSI activities and to find out what was on the political radar 
screen for education. 

Nearly every SSI established a steering committee or major board that included highly-placed 
stakeholders in the state. In terms of political strategizing, involving these key power-brokers 
was imperative. One way to involve the needed state-level stakeholders was to tie the SSI to an 
existing group in the state that was seen as influential. The South Carolina SSI crafted its design 
after a report issued from the Mathematics and Science Advisory Board (MSAB), a highly 
visible and broadly-based group appointed by the governor and state superintendent. The SSI 
was sponsored by the MSAB as a new entity within the Department of Education, virtually 
assuring the support of key state leaders in government, higher education, business, and K-12 
education. The support of the Board and of the State Superintendent persuaded other important 
constituencies to support the SSI as well. 

Those SSIs that tied their initiatives to existing reform movements initiated by key players in the 
education system also garnered early support of key stakeholders. Connecticut, Delaware, 
Michigan, Puerto Rico, Texas, and Vermont are examples. Hitching the SSI to a reform being 
supported at the highest levels, even where the specific initiatives were something new, lent the 
SSI the support of those driving the reform. Governors and state superintendents again were 
frequently key players, and business and industry leaders were often solidly behind these reform 
movements as well. 

Some SSIs crafted specific components to respond to the interests of key state-level stakeholders 
as a way to engage them more fully in the reform. Michigan expanded a component targeting 
pre-service education due to the input of influential contributors to the SSI. Maine appointed key 
business leaders to head its community outreach and systemic planning components. 

Connecticut also crafted its plan specifically to respond to the interests of key stakeholders, 
expanding efforts in pre-service education like Michigan, and seeking specific ways to engage 
stakeholders like Maine. An important reason for these changes was Connecticut’s early 
commitment to begin the SSI as part of a twelve- year plan for reform of mathematics and science 
education in the state. The PI reflected: 

Some people said, “ You know, we ’re never going to get people to stick around for twelve 
years. ” Well, no, [unless] they have a piece that they can play, not necessarily money 
that they get, and they can begin to see that we as a state are making movements towards 
our combined goals. People have all kinds of longevity. 

Another effort some SSIs made, specifically to address equity goals of the initiatives, was to 
include at the highest levels of the SSI management and operations representatives of 
traditionally underrepresented groups. The Delaware SSI made a point to involve scientists, 



Horizon Research, Inc. 



48 



March 2003 




mathematicians, science and mathematics educators, and administrators from Delaware State 
University, and also included community organizations serving historically underrepresented 
children, such as the Boys Club of Delaware and Big Brothers/Big Sisters. Similarly, the 
Georgia SSI sought to establish reform centers around the state housed in universities serving 
specific regions, including some serving primarily minority populations, as well as engaging an 
ethnically-diverse leadership team. 

The importance of these efforts to include diverse, powerful stakeholders in the state had 
important effects not only on the SSIs, but on the interest and involvement of these constituencies 
in mathematics and science reform in general. One leader in Maine recalled: 

[We] were bringing groups of people together that had never been together before, and 
talking about what was important about math and science education in Maine for cdl 
students, and where we were and what could be done, and there was just such enormous 
leverage in that. 

A leader in Michigan offered a similar description: 

The [Michigan] SSI provided a way to convene people, which hadn ’t been happening. 
There were all these pieces out there, but no entity to convene the players. So they got 
people together and got them talking in the same ways and the vision emerged from that 
work. 

One challenge that several SSI leaders specifically noted was convening leadership in the states 
in the field of science education. Some SSIs did quite well in this regard, such as Ohio, Georgia, 
and Puerto Rico, all of which used scientists/science educators as Pis. Two factors seemed to 
create challenges in this area. First, science was not supported as widely as mathematics in state 
policies. Also, across states, SSI leaders variously described leadership in science education as 
“fractured,” “fragmented,” and “Balkanized,” largely along discipline-based lines. 

Another challenge was negotiating turf battles over funding. Several SSI leaders noted that 
representatives of existing projects often wanted to see the SSI funds divided among projects as a 
way to sustain or scale-up existing work, rather than have the funds centralized as a way to 
coordinate new and existing work toward common goals. The SSIs’ attempts to leverage 
existing funds, such as Eisenhower or Title I, could also cause turf battles, since many people in 
the state viewed these sources as rightful funding for their projects. Two SSI leaders, from 
different states, offered similar thoughts: 

Well there were some people who were a part of their own program or project who felt 
very strongly about it and wanted it to be funded. I'll say in a very biased way there 
were a few groups like that that really didn ’t understand the systems approach, and what 
they wanted was money for their own particular project. A couple of others still 
remained pretty negative about the SSI, but that’s a small number and it’s clearly 
isolated to particular project efforts. Some of them thought that they should get some of 
the money for their project. (SSI Project Director) 
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The thought that a lot of money would be dropped in one bucket with a single set of 
choices was problematic. Some people... were looking for dividing the big pot of money 
into little pots to give them money for the truly excellent things they had been doing. But 
that wasn 7 systemic. This wasn 7 just a lump of money to improve science and 
mathematics education around the state. It was a systemic initiative, and that was 
something brand new. (SSI Principal Investigator) 

A concern some leaders noted was that highly influential individuals, especially from higher 
education, often viewed their work as being of national or even international significance, and 
looked at a state -based SSI as diminishing the scope of their work, rather than expanding it. As 
one SSI leader put it: 

SSIs are team efforts. If you ’re a star, going back to play in the band isn 7 much fun. The 
problem was that there were excellent people out there and they were concerned about 
losing control and losing their spotlight. 

Involving Key Regional and Local Stakeholders 

In addition to involving influential state-level stakeholders in their initiatives, most of the SSIs 
viewed the engagement of regional and local stakeholders as critical, especially in the 
predominant policy environment that increasingly favored local decision-making about how to 
pursue state education goals and mandates. Explained an evaluator of one of the SSIs about the 
importance of involving the “right” people at the local level: 

You have to have a certain set of people on the team. In the districts where there was a 
good mix of teachers, principals, community members, you saw a lot more progress. But 
if you didn 7 have some policy makers on your team, you could go back and talk until you 
were blue and not get anywhere. 

School-level administrators were considered particularly key. Said one PI of their efforts to 
improve classroom practice, “at the institutes, people would get excited, but if there was no 
leadership back in the school, it would be gone in six or seven weeks after the summer was 
over.” Recognizing the problem was important, but even so getting support from school and 
district administrators was sometimes more difficult than the SSIs had anticipated. Said one PI: 

It was hard to arouse the interest of principals, superintendents. The one success we had 
with that was probably with the district science and mathematics coordinators, where 
they were highly cooperative, but you ’d expect that. We just were unable to go beyond it. 

The Vermont and Delaware SSIs, among others, made specific efforts to take their messages out 
to the regions and localities in their states through series of meetings, conferences, and dialogues. 
Other SSIs formed regional and local advisory and leadership groups that mirrored the state 
steering committees and boards to foster local commitment to the reform. 

One of the challenges the SSIs faced was to find the appropriate balance between their statewide 
and local “presences,” getting the most leverage out of each. South Carolina’s SSI was 
particularly adept at maintaining its statewide presence, and simultaneously using a regional 
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structure to foster buy-in to the initiative. Designed as a way to provide equity of access to 
resources and services, the SSI established thirteen regional HUBs spread across the state. Each 
HUB had an advisory board that included the superintendents of all constituent districts as well 
as other local business and community leaders. To local leaders, the SSI was their own HUB, a 
nearby source of expertise and resources previously unavailable to many districts, and over 
which they had some control. 

Recognizing the importance of involving district superintendents in its regional advisory groups, 
but also knowing their tendency to send representatives to meetings, the South Carolina SSI 
deliberately created two separate groups in each region. One group included the superintendent 
of every district in the region, but met only annually. The other included more of the district 
“worker bees” and met quarterly, with the understanding that they would keep their 
superintendents informed. 

As their activities unfolded, more and more SSIs came to realize the importance of getting buy-in 
from key stakeholders, and created mechanisms for doing so. Reaching out to regional and local 
leaders in the state presented a particular challenge for some SSIs, however, especially in the case 
of states that had existing regional education structures. Although it seemed both necessary and 
natural to work through existing regional service centers or education service units, many SSIs 
encountered resistance in these efforts. For example, one leader in Michigan described the 
interface between the SSI and the existing regional centers, “Very early on, the SSI said these 
Mathematics and Science Centers are a critical piece of this thing. If we’re ever going to make 
anything happen, we’ve got to maintain this piece of infrastructure. The SSI went to bat 
frequently for the Centers to get funding for them, but the Centers saw the SSI as a competitor.” 
The Michigan SSI recruited the directors of the regional centers to participate on the initiative’s 
management team, but tensions remained throughout the initiative. 

The Arkansas SSI also encountered friction with the state’s regional cooperatives. The key issue 
was funding. Two SSI leaders said: 

And there was some tension between the SSIs and [the Cooperatives ]. And part of it was 
pride of ownership within the regional sendee centers. Because they were doing a lot of 
one time in-service, short term and so forth. And anything that cut into their deal, they 
felt totally threatened by. 

They ’re all ... run very independently, even of the state department of education. . . . Once 
we got the grant, many of them were actively hostile because they thought we wanted to 
use “their” money for math. 

Receiving federal money from NSF that was not apportioned among existing regional entities 
caused some difficulties, but attempts to capture or leverage other funding within the state — a 
charge that the SSIs accepted and pursued — likely put the initiatives in an even more delicate 
position. It seemed almost inevitable that in most cases, some group within the state would view 
the SSI as a competitor for resources. 
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Establishing Neutral Political Ground 

Across the SSIs it was frequently cited as important that the initiative be seen as providing 
somewhat neutral political ground for debates and decisions about mathematics and science 
education. The SSI’s organizational home and its chosen role in reform were important factors in 
establishing the SSI as “neutral turf’ from which the initiative could cultivate understanding of 
and support for its vision. 

As noted earlier, some SSIs chose to establish the organizational home of the initiative in an 
independent organization. For these SSIs, establishing politically neutral territory for the SSI 
was perhaps the easiest. For example one Co-PI said of the Maine SSI: 

The importance of the SSI in Maine choosing to go to a non-profit structure was very 
valuable, because it allowed that group to be flexible, to shift and change, and not 
necessarily be politically aligned anywhere. It is a high risk strategy in the sense that we 
could be disregarded, but you aren 7 pigeonholed anywhere. So you can take the high 
road without having to get dragged down by other values, you know, in the particular 
institution. ... I think the insight of starting this organization was very good. 

The Connecticut SSI also housed its initiative in an independent, non-profit organization. Two 
leaders there had very similar reflections on the importance of this decision: 

And I think the Academy leveraged itself and it really became and is a gathering place ... 
It’s not a coffee house, but it became a place where lots of people could come together 
with different, disparate interests and agendas and were able to sit down and talk and do 
some common planning and do a common approach to attacking problems. (SSI Co-PI) 

I think it was very important to have it outside the Department. We were very pleased 
with that model. It’s just they had more clout with business and industry. They had more 
clout with parents; parents are sometimes very distrustful of departments of education, 
and so this was an outside entity that kind of catered to them, so I think that was a good 
piece. (SSI Co-PI) 

Although some states housed their initiatives in universities or state agencies, the need to be 
viewed as politically neutral in the state remained important. For example, the New Jersey SSI 
highlighted in its initial proposal that the reform movement in the state existed independent of the 
leaders of the SSI, and that a key goal of the initiative would be to build an infrastructure for the 
already- supported reform effort, and to foster bipartisan support for the reform so that it could be 
sustained through changes in administration and beyond NSF funding. 

Similarly, the South Carolina SSI embedded their efforts within an existing movement toward 
reform in the state. Despite being housed within the state Department of Education, having tied 
the SSI closely to the recommendations and education reform plans of the Governor’s 
Mathematics and Science Advisory Board (MSAB), the initiative was not viewed as enmeshed in 
the politics of the Department. The MSAB survived changes in state administration, and 
provided a sense of political neutrality and independence for the reform and for the SSI. 
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Another important reason to seek some political independence from certain bureaucracies in the 
states was the need for the SSIs to move quickly when opportunities arose or when mid-course 
alterations were required. Also, state agencies often carried a history or reputation that could 
unnecessarily burden the SSL Again, the choice to house the SSI in an independent, non-profit 
organization was often helpful in this regard. A leader in the Vermont SSI described how the 
initiative came to its decision: 

One reason is that many people, particularly in the business community but also in the 
K-12 community, wanted to make sure that the organization would be fairly agile. 
Neither our higher education system nor our Department of Education is particularly 
known for their agility. It takes a long time to get things done. There was a sense of 
urgency to this, and it wouldn ’t happen in either of these two bureaucracies. There was 
a fairly strong push to make [the SSI] a nonprofit organization. The second factor was 
that our Department of Education in Vermont was not strong at that time. In some ways, 
compared to some states, it still is not. It was not perceived as a place to put your 
money. So the thinking was to keep it non-governmental, but also non-academic in its 
organizational structure. 

Very similar sentiments were expressed by a key player in the Connecticut SSI who was on the 
staff at the Department of Education: 

[The leaders ] very quickly realized that it ought to be something that was outside of the 
Department. In terms if you think about the decision-making, because we’re a public 
agency, and everything else, the kinds of decisions we make, how it has to be cleared, is 
much more complex than if you can go outside into a non-profit, quasi-public 
organization. 

Again, those SSIs that housed their initiatives in public agencies highlighted similar concerns. 

For example, the Arkansas SSI was developed as a partnership between the Department of 
Higher Education and the Department of Education. The choice of fiscal agent for the initiative, 
however, hinged on the need to make decisions quickly and to avoid crippling political 
difficulties. A leader in the Department of Education described the choice to make the 
Department of Higher Education the fiscal agent: 

That’s a very small agency up there and things can happen like that (snaps fingers). You 
know here we’re 350 people, and this is one of a multitude of things that’s going on up 
here. Even though it was a priority for some of us, there were others in the food chain, 
that this was not the top priority. ... It’s where it had to be to keep it out of the politics of 
this building. 

In Texas, the SSI was set in a university, but noted in its proposal for Phase II funding: 

While connecting people to TEKS-compatible practices and innovations, the SSI does not 
promote any single solution, program, approach, or ideology. The SSI insists only on 
respecting state education law, which has enshrined equity and the TEKS as the basis of 
the Texas education system. This approach has established the SSI as a neutral party 
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and catalyst for change. It has enabled the SSI to bring together in a common enterprise 
individuals with diverse perspectives and political commitments. 

The Texas SSI used an innovative strategy to engage leadership around the state, to maintain a 
high statewide profile, and to make the work of the SSI as politically inclusive and neutral as 
possible. Involving stakeholders in substantive work of the reform was one of its primary 
missions. The SSI committed itself to having diverse viewpoints represented around particular 
issues in mathematics and science education, with the SSI playing something of a background 
role in order to facilitate buy-in to the work of reform: 

We knew that if we could become the convener, we woulcln 7 have to be the leader, if you 
will, because we could facilitate and lead through facilitation and consensus building, 
because Texas is a very independent state and it’s kind of hard to be the leader. There 
are too many leaders and they’re not willing to give up their power and authority, so 
that’s the strategy we’ve used and it’s worked pretty well. It’s worked better in math 
than in science, science is still a bit fragmented, but we’re beginning to build a coedition. 
So that was part of what we were thinking about way back there. We wanted to be the 
convener, the consensus builder, around the state, and I think people would say that we 
are at this point. 

The Texas SSI made a concerted effort to involve leaders from both progressive and conservative 
political and educational groups in creating and supporting a common vision. The idea of 
building networks of leaders across the state with an interest in addressing a particular issue was 
operationalized for several years in the Texas SSI with its “Action Team” strategy, and has 
continued in more informal ways recently. Each Action Team included a member of the SSI’s 
Board of key power brokers in the state, which ensured a high political profile for the work, and, 
in response to evaluative feedback to its early efforts, an SSI staff member, which ensured that 
the work was progressing and remained aligned with the overall reform vision. The Project 
Director described the basic strategy, using the example of teacher preparation in mathematics: 

We did a “state of the state” [research report] on whatever the issue was; let’s say 
preparation of elementary teachers in mathematics. ... From that report we developed 
an Action Team. ... We talked to people to find out who the leaders were and we talked 
to the leaders to find out who the other leaders were. So we did kind of a sociogram of 
who the leadership was in the state. . . . We brought those people together and we 
continue to try and identify new and emerging leaders as they come on the scene. And . . . 
now ... we’re beginning to get into the business of supporting the development of leaders. 
Early on it was a matter of networking and finding out who was respected and finding 
out who people pend attention to. 

Focusing on one issue at a time allowed the SSI to identify vested groups of stakeholders 
throughout the state in order to involve them in substantive work. Concentrating on issues, as 
opposed to specific interventions, also allowed the SSI to include on the Action Teams people 
with diverse viewpoints. 
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The first task was creating a broader political base from math and science education 
than NCTM or N ST A were able to establish by themselves. So the idea of localism and 
populism requires that you have a really diverse and multi-dimensionally diverse 
leadership, and the Action Team model turned out to be really good for Texas.... [The 
SSI] was able to go out and reach out to the Saxon people, and to the direct instruction 
people . . . and I had very cathartic meetings with the NCTM and . . . NSTA people, about 
the fact that the SSI was not going to be solely national standards based, but it was going 
to in fact draw in the people who were teaching kids. So we had ... meetings with ... 
some of the more conservative political groups that were interested in local control as a 
principle. ... Building a more politically diverse leadership for math and science, that 
was really hard. 

Using the approach of convening diverse groups of leaders around issues has led to some 
important lessons in the Texas SSI about creating and supporting a shared vision for reform. 

First, although the strategy enables broad participation within and across issues, it requires 
consistent leadership from the SSL The Texas SSI made a point to hire respected mathematics 
and science education leaders in the state, capitalizing in particular on the downsizing of the 
Texas Education Agency and connections to more conservative education groups, as well. 
Second, the SSI learned the importance of assigning responsibility for the voluntary work of 
actions teams to paid SSI staff leaders, who could keep the work moving ahead in alignment with 
the vision. Third, it was important reach out to those who played key roles around issues in the 
state, whether as policy-makers or local service providers, and include their voices. For this 
reason, the expansive model of the Action Teams allowed the SSI to identify and include local 
leadership as well as state leadership. 

Reaching Out to Public Audiences 

Another major decision the SSI leaders had to make was how to get the word out about their 
initiative in order to garner public attention and ultimately to gain support. SSIs were required to 
address this component in the program solicitation, which specified, “Each proposal must 
include significant efforts to increase public awareness and understanding of the need to support 
improvements in science, mathematics, and engineering education” (NSF, 1990). Many SSIs 
recognized the importance of parent and community buy-in, increasingly so as challenges to 
reform curriculum and instructional materials became more visible nationally. However, people 
with expertise in mathematics and science education often did not have expertise in garnering 
community support, and even if they had the know-how, SSI resources did not permit large 
public relations campaigns. One PI talked about not only the financial costs, but also the 
“political costs” of diverting resources from professional development to public relations, “We 
did not put our money into what I would call public relations types of activities. They were 
incredibly expensive and we would have been killed locally within the state if those dollars had 
been going to PR rather than to teachers.” 

Smaller efforts were more common than large scale public relations campaigns, including SSI 
leaders presenting at community forums; conducting Family Math/Science Nights; and 
distributing pamphlets about the goals of the reform, and how parents could help support their 
children’s learning in mathematics and science. 
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The Project Director in Arkansas expressed the importance of having a strategy for making the 
SSI and its vision visible in the state, “[You have] to get the product out there in six months or 
less, [and have] good publicity or people wonder what you’re doing.” A few strategies were 
employed for getting the word out about the Arkansas SSL One was centering the initiative on a 
signature program of statewide professional development called “Crusades.” Nearly everyone in 
Arkansas involved in mathematics and science education knew of the Crusades, and other 
interventions, such as professional support for administrators, were built around the Crusades. 
Another way of “getting the word out” was to hire a Communications Director, who facilitated 
communication among the key partners of the initiative, the Arkansas Departments of Education 
and Higher Education, coordination of a Family Math and Science program and EQUALS and 
ESL EQUALS programs for the neediest districts, and publication and dissemination of 
“booklets” to spread information to various audiences about mathematics and science reform. 

In many SSIs the approach to making the SSI and its vision visible in the state combined a broad 
public awareness campaign with targeted strategies for reaching parents. In general, the main 
purpose of the public awareness campaign was to cultivate understanding of the importance of a 
high quality science and mathematics education for all students. Strategies for reaching parents 
generally tried to educate them about the kinds of mathematics and science teaching and learning 
the SSIs advocated. 

Establishing partnerships with stakeholders has been a vital aspect of both general public 
awareness and targeted parent outreach campaigns. For example, the New Jersey SSI included a 
very broad array of partners in its work and used its close ties to business in the state to present 
statewide public events, to conduct outreach to parents and communities reliant on different 
business sectors, and to influence key power brokers through the state’s most influential business 
leaders. The Georgia SSI developed relationships with the Georgia Partnership for Excellence in 
Education (founded by the Georgia Chamber of Commerce and the Georgia Economic 
Developers Association), Georgia Public Television, and the Georgia School Boards Association 
to convene town hall meetings and a statewide Governor’s Conference to develop a recognition 
of the importance of ensuring high quality mathematics and science education for all students. 
Coupled with this broad campaign was a project targeting parent education through Albany State 
College, which developed take-home activities and workshops so that parents, especially of high- 
risk children, would understand the nature of the science and mathematics teaching and learning 
the SSI advocated. 

Although many SSIs did put resources and effort into public outreach, it was particularly difficult 
for the initiatives to track the impact of these activities. Leaders in the Connecticut SSI viewed 
changes in many districts’ approach to summer school, moving from remediation to enrichment, 
as strongly connected to the initiative’s public outreach through the popular summer program 
Learning Doesn ’t Take a Vacation and its widely disseminated monograph The Case for More 
Time and Better Use of Time in Connecticut Schools. In most cases though, public outreach 
efforts were viewed as important, but difficult to measure in terms of impact. One leader in 
Maine commented: 

One area that we ... struggled with ...is the evaluation and assessment of the 

effectiveness of community activities. ...It seemed logical and intrinsiccdly good that 
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that would over time have an impact, but I wasn 7 able to say that . . . community 
activities, while appearing to be inherently good on the surface, ultimately changed any 
of the [assessment] results one point one way or the other. We were never able to get 
there. We were a third order effect. I think part of the work that we did allowed and 
greased the skids for the Learning Results. / think it opened to door for people being 
willing to talk to educators about the value of education. 1 think it did a lot of great 
things but it was never able to get that far. 

Many leaders remained convinced, though, of the importance of public outreach activities as a 
means to create fertile ground for the deeper changes the SSI hoped to seed throughout the state. 
In particular, support for curriculum standards, increased graduation requirements in mathematics 
and science for all students, and classroom instruction focused on conceptual understanding were 
seen as potentially related to public outreach efforts of the SSIs. 

Using Results to Build Support 

The involvement of key stakeholders in decision-making and widespread understanding of what 
the reform was intended to accomplish were both important. More important still was the ability 
to show stakeholders that the reform strategies of the SSI produced valuable outcomes, especially 
in terms of student learning. 

When an SSI was able to provide credible evidence of impact, they got a lot of mileage out of it. 
In some cases, evaluation data were disseminated through reports to funders and stakeholders, as 
well as newsletters to businesses and the community. The Ohio SSI prepared a “Pocket 
Panorama” with bar graphs showing a narrowing of performance gaps between white and 
minority students, based on administration of a test comprised of NAEP released items, and 
linked to the SSI interventions. Distributed to key individuals in the state legislature, the results 
greatly increased policy maker interest in the SSI. A PI in Ohio said, “We began collecting 
student data . . . and began our public relations campaign with the ‘Pocket Panoramas.’ They 
went out to every legislator and every superintendent. We had to build a constituency. ... You 
know to keep funding you have to show them kids are learning.” 

The Puerto Rico SSI used its evaluation in a similar fashion. Armed with results showing clear 
evidence of improved student performance, SSI staff began to “persuade” others of the value of 
reform. According to one person, the Puerto Rico Department of Education “bought into reform 
because over time they saw more and more evidence that SSI schools were doing better than 
other schools.” 

The Vermont and South Carolina SSIs were also active in publicizing and communicating not 
only what the initiatives were doing, but also what impacts the SSIs were having. In both states, 
these activities were viewed as critical to continued and growing support for the initiatives, 
particularly in terms of leveraging funds to sustain and scale-up efforts. 

An important consideration in using student learning results was discussed by one leader in the 
Texas SSI. Accountability is an extremely powerful force in education in Texas, and the state 
has made a considerable investment in a data collection system. The SSI has used student results 
from this system to seek support for the initiative because it is both inexpensive to do so, and 
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because it is a system that leaders in government and education understand and pay close 
attention to. At the same time, the SSI has sought to compile and have districts collect student 
outcome data using assessments independent of the Texas accountability system. This strategy 
has been utilized because the powerful business community, although “solidly behind the 
accountability system,” is more convinced by data showing impacts on student learning from 
sources independent of the tests that had been created to drive the reform. 

Finally, political sensitivities have arisen in trying to attribute gains to the SSI when other entities 
in the state were also working toward the same goals. An important lesson leaders in one state 
relayed was that in order to grow support for the reform vision, those who have contributed to it 
need to be recognized and appreciated: 

Not early on, but after 1995, we became known as the group who could deliver in the 
state. The State Education Department began to get a reputation as being useless. I 
could not have that happen, because they were an important partner. I had to work very 
hard with superintendents and others in the state that the Department was a vital partner 
in our work, and what we were doing was in cooperation with the Department. The 
Commissioner . . . and I began appearing a lot together at conferences to talk about the 
work we were doing and supporting each other. 

The Project Director of the Texas SSI communicated a similar rationale for sharing credit for 
successes: 

Actually I think if you ask around right now, although our name will be mentioned . . . 
nobody would say [the SSI] did it. And one of our tenets, if you will, is if you ’re really 
doing systemic work, everybody can take credit and they will be right, and so that’s kind 
of how we measure it, is it embedded in the system. If everybody is saying that we did it, 
then it’s in the system. If it’s only one group that is advocating for it, then it’s in danger. 

Many SSIs reported having to walk a thin line between attributing improvements to the SSI and 
sharing credit throughout the state. Emphasizing the collective nature of the statewide efforts 
remained critical for maintaining partnerships vital to the initiative’s continued progress. 
However, illustrating the tangible and growing results of the state reform with attribution to the 
SSI was necessary to make a case that the initiative was an important contributor to the statewide 
education system and to demonstrate progress to NSF. In addition, as the SSIs worked to 
institutionalize and embed pieces of their work within the structures and functioning of the 
education system, continuing their work on systems change required that stakeholders in the state 
view the initiative as a unique and valued contributor to improvements. 

In Maine, at the conclusion of NSF funding and while facing the disappointment of having not 
received continuation funding, the leaders of the SSI contemplated whether to move forward as 
an entity or to disband. The Project Director described how the initiative resolved the issue: 

One of the things that I said . . . was that 1 don ’t want to be the group that is just going to 
sort of peter out, you know, that each year something was going to get tighter and tighter, 
and . . . that essentially we should just say, “Hey, we ’re done. We did all that we could 
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and we’re really pleased with what we did. ” Or we should say, “This group, the [SSI] 
has value. It’s adding value to the state and it’s a good contribution and we should keep 
it. ” And on that basis we could go forward in a positive way. And the board decided that 
there was absolutely a need and that there was important value. 

Having reaffirmed commitment to the SSI from within, the Project Director knew maintaining 
strong political support was also critical to sustaining the reform: 

7 went to talk to the Department, the Commissioner at the time, and said basically the 
same thing, “I don ’t think it is worth the state having this group that just sort of peters 
out at the end of the funding. It would be much better to either end it or ... make a 
commitment to it. ” And he said, “Well, my commitment is to it. ” 

Other SSIs have taken a similar tack with districts that have used their services. Arkansas, 
Connecticut, Ohio, Texas, and Vermont, among others, have moved to models in which they 
provide their tools and services for a fee. Districts, for their part, have discretionary money to 
support school improvement and professional development, and, as a leader in Texas said, “If 
what we do is of value we believe people will pay for it, and that’s a major piece of our 
sustainability plan.” The Connecticut SSI PI shared this thinking: 

We have simply decided that we ’re not going to chase money, what we ’re going to do is 
to continue to do really, really, fine work. And if our work product doesn ’t speak loud 
enough then we should go out of business, that’s what I've convinced [our] Board. 

Increasing School and District Leadership Commitment to Reform 

Just as the national trend toward more centralized and more powerful state guidance and 
accountability policies led most of the SSIs into policy work, the concurrent trend toward 
decentralized decision-making created needs for developing reform leadership at the local level. 
SSIs in states that traditionally had locally controlled decision-making faced a long-standing 
history of district and school leaders used to independence in administrative and instructional 
matters. In a number of SSI states that traditionally had centralized control over decision- 
making, local leaders were increasingly being given authority in important decisions. A key 
challenge for the reform was to educate local leaders so that they would make decisions that were 
consistent with the reform vision and tailored appropriately to their context. 

Nearly all SSIs began with some effort to work with school and/or district administrators. The 
Maine SSI’s “Beacon Schools” would include administrators as a part of the decision-making 
team charged with creating models of reform at the school level. The SSI in Georgia planned to 
develop the Program for Administrator Support of Science and Mathematics through one of its 
Professional Development Schools for delivery through the state department’s Leadership 
Academy. Delaware took a grassroots approach of fostering culture-based change in schools, 
including working with school administrators as well as faculties and communities. New Jersey 
planned to deliver long-term and intensive professional development to leadership teams from 
schools and districts that included teachers and administrators. 
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Increasing Administrator Awareness of the Reform Vision 

As the SSIs proceeded, the key role of administrative leadership became more evident. For 
example, in Nebraska one of the SSI Pis recalled that the regional partnerships supported by the 
SSI worked well and scaled-up efforts “well beyond the boundaries of the grant” mainly in 
districts where the superintendent and principals demonstrated a solid commitment to the reform. 
In Vermont, one leader recalled: 

Very early there was a lot of focus on building the capacity of teachers, not only to teach 
well but also to lead the charge, if you will. One of the mistakes in those early days is we 
didn ’t recognize how important the role of formed leadership was, and in many cases we 
created some high powered teachers ready to lead in their schools and their principcds or 
superintendents weren ’t on-board at all. I think we learned from that, but it was an issue 
early on. 

Similarly, in Michigan, which devoted considerable resources to working with the neediest 
districts in the state, designated Focus Districts, the SSI leaders found that district leadership was 
a key to success. The SSI evaluator said, “By the second or so year into funding for the Focus 
Districts, the SSI formed a superintendents group. It was quite obvious that where the 
superintendent was involved, it made a difference.” This realization was especially troubling to 
Michigan SSI leaders, because the Focus Districts that were not progressing as hoped needed 
much more support than the SSI had the resources to provide, particularly in the area of 
administrator leadership development to support reform. 

The support of administrative leadership was considered so important to the Arkansas SSI 
leaders that its statewide professional development effort was redirected in later years of the 
initiative to “focus on schools where the leaders had been through the [SSI’s] Leadership 
Academy, or were willing to go through it.” The SSI leaders undertook this step with some 
serious reservations, because the notion of working primarily in districts with this level of 
readiness for reform could mean leaving some of the districts with the greatest needs behind. 

One staff member said, “Equity doesn’t mean that everybody gets equal treatment. Equity means 
that we reach out for potential and we nurture it.” The decision to work with districts that 
demonstrated administrative commitment could be seen a way to “reach out for potential and 
nurture it,” but the difficulty the SSI already had in encouraging minority participation in its 
activities was of particular concern. 

Those SSIs that focused on school/district administrators from the beginning of their initiatives, 
as well as those that began to invest more resources in developing administrative leadership, 
often targeted two specific objectives: evaluation of classroom instruction and adoption of 
instructional materials. Many SSI leaders reasoned that a key leadership role of school principals 
influencing classroom instruction was their teacher evaluation functions. A Connecticut SSI PI 
described the concern and the SSI’s response: 

Another thing that we recognized in that same time frame was that all the work up to that 
point in time, in the majority in the country, had been on teachers. What we recognized 
was that administrators needed to be knowledgeable about what good math and science 
education would look like. ... What we found was that the majority of principals that 
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were doing it, evaluation, didn ’t know what good mathematics instruction looked like, 
because they had been taught mathematics in a way that worked for the bell- curve 
mentality; it doesn ’t work for all-children-succeeding mentality. So we began working to 
develop means of providing professional development to administrators on what this 
mathematics looks like, [and] science. 

It was also evident to SSI leaders over time that instructional materials exerted a heavy influence 
over classroom practice, and that in many states local administrators had considerable discretion 
in instructional materials adoption. Depending on the state, other local leaders, such as teachers 
in leadership positions or curriculum supervisors, also played key roles in materials adoption. In 
several states the SSI worked with these local leaders specifically around this task. The Texas 
SSI invested heavily in professional development and tools designed to aid principals in selecting 
mathematics and science instructional materials aligned with the state standards. In South 
Carolina, the statewide science materials adoption was used as an opportunity to have the 
regional HUBs work directly with the committees in districts appointed to this task, with a focus 
on adopting high quality materials. The Georgia SSI also took advantage of timing and 
circumstances to work directly on the materials adoption function of local leaders using the SSI’s 
Framework for Learning Mathematics and Science, as the PI described: 

It just so happened that the Framework was [reviewed] by the state board and then it 
was right in a science adoption year for materials. So we really worked with district 
science coordinators to get them to implement the use of the Framework in selection of 
materials. They really thought it was an extremely useful tool. 

The importance of administrative leadership support for reform not only became increasingly 
recognized and addressed as the SSIs matured, but the SSI leaders’ view of the range of 
leadership functions relevant to support systemic reform broadened. A point of access to many 
vital leadership functions, several SSIs found, was a common move toward improvement 
planning and data-informed decision-making at the district and school levels. SSIs adopted two 
main strategies for developing local leadership to capitalize on this trend. First, some SSIs 
repositioned themselves as technical assistance providers to districts and schools to aid in data- 
informed decision-making. Second, some SSIs developed tools for districts and schools to use 
for needs assessments and strategic planning, and taught local leaders how to use these tools. 

The South Carolina and Vermont SSIs followed the wave of legislative acts in their states 
requiring districts and schools to provide reviews and plans to the state, setting the initiative up 
as an aid to local leaders faced with responding to new requirements. The South Carolina SSI 
developed toolkits to aid local leaders and provided assistance to schools, “clusters” of schools 
that formed K-12 feeder patterns, and districts to use their tools to analyze and plan for teacher 
and administrator professional development needs and articulation of curriculum across grades. 
Although much of this activity moved the initiative away from its direct work around 
mathematics and science, the SSI leaders saw it as a powerful opportunity to bring coherence to 
key decisions affecting mathematics and science education. 
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In Vermont, working with local leaders around broad issues in science and mathematics 
education as a way to broaden and deepen change had become an important focus of the initiative 
over time, as one leader described: 

One of the most important things the Teacher Associates learned was how to work with 
the science committee in the school, get them to meet effectively on a regular basis and 
continue to pursue changing science education in that building. That was an important 
element to making change in the building, and drew more people in than we ever had 
before. Now we had, particularly in our partnership schools, active science and math 
committees looking at their curriculum. So a lot of curriculum change started to happen, 
to incorporate the new science and math materials that were available. So that strategy 
pulled in a lot more people who would participate on a continuing basis than what we 
had with the summer institutes. 

The SSI has more recently altered its leadership development strategies to fit a new policy 
context in the state: 

Act 60 set up expectations for schools that were different from what had been. 

One was that schools would start using data in more serious ways, that schools 
would examine their student performance data in context of other data and create 
an annual action plan to focus professional development, resource allocation, 
energy, etc. That’s made a huge difference. What we’ve had to do as an 
organization is to become responsive to the action plans. If schools have decided 
where they ’re going to put their emphasis and spend their time in professional 
development, we need to be part of helping them get what they need. So we 
spend a lot more time helping people do data analysis, translating the data into 
action plans, and then implementing those plans. That has fundamentally 
changed the way we work. 

Where previously the SSI would have offered institutes, and waited to see who came, they are 
now more likely to develop intervention to meet the needs that groups of schools have identified. 

Providing Technical Assistance to Local Leaders 

As the SSIs progressed, they increasingly moved beyond awareness goals for administrators to 
providing technical assistance to schools and districts in strategic planning for mathematics and 
science education reform. For example, the Connecticut SSI produced and disseminated 
documents and tools to help districts conduct needs assessments and strategic planning around 
their mathematics programs, science programs, uses of educational technology, allocations of 
time, and their broader systems of policy, resource allocation, and management. The 
mathematics and science consultants at the Department of Education both identified these tools 
as not only key successes of the SSI, but important resources for districts in the state to align 
their mathematics and science programs with state expectations, and to improve their programs 
and systems as a whole. The PI said: 

First of all, we began to recognize that we needed a richer array of data, and about 1993 
to 1994 we really became strong proponents of data driven decision-making.... What we 
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wanted to do is turn the data into information, and not so much data for information for 
the outside world, but we wanted to turn it into information that the educators themselves 
could use to help set goals, to help drive planning, to help measure achievement against 
goals, and we also wanted them to say, there is more in the issue of data, there’s more 
than just student performance data. There ’s a lot of performance information about the 
system itself that needs to be collected, because lots of things have an impact on student 
performance. So the use of information to guide improvement within the district, as 
opposed to using test data to bang over teachers ’ heads, has become a very strong 
component of what we do, and we continue to do it more and more and more. 

...Then we started seeing that there were very few tools out there that would help a 
school system look at itself, and measure the degree of alignment it had among a bunch of 
these different elements that we had identified. You have all the policy, you have practice 
in the classroom, you have school board policy, you have national standards, I mean, you 
have partnerships, your grants, all these different things. How do you measure in a 
school district of 30,000 or 300, how well that alignment is taking place? So we began 
working on developing tools in that direction as well. 

The SSI hired a staff and developed a cadre of consultants to provide technical assistance to 
districts that make a three year commitment to work with the initiative. The SSI’s tools are a 
starting point not only for districts to identify their own strengths and challenges, but also to 
focus on the kind of systems improvements advocated by the SSI, and to measure progress over 
three or more years. One SSI leader also noted that the use of the tools is a way of assuring that 
the work of the initiative’s consultants remains focused on the SSI’s reform vision. 

The Texas SSI took a very similar approach, creating a series of leadership development 
workshops for administrators and other local leaders as it moved into its Phase II funding. The 
SSI has also created tools for local leaders to use, disseminated these tools widely through its 
networks and communication mechanisms, and supported a number of districts in the use of 
these tools. One leader in Texas described the rationale for this approach in relation to the Texas 
Essential Knowledge and Skills (TEKS): 

We realized that the principal and the centred office staff and the teacher leaders, the 
superintendent, they all play an important role in supporting teachers in implementing 
the TEKS, so we built this series and it now has four two-day seminars where they learn 
how to use the tools that then are used with teachers to help them understand the TEKS, 
to help them understand the assessment of the TEKS in their classrooms, how to use their 
assessment to guide instruction, the kind of instruction that’s important and on and on. 

... It’s a whole series that we ’re continuing to build. We ' ve built a companion piece for 
algebra, and we’re thinking about others. We built a companion piece for advanced 
placement; that’s an important part of our system here. For vertical teams we ’ve built a 
companion piece now. Then we started to build a whole set of assessments; it’s what 
districts are understanding, as they can ’t wait for the annual assessment. They need 
formative assessment to give feedback to teachers about how kids are doing, ...so we’re 
building a whole set of assessments as well. The important part of this is that we make 
these resources available to these leaders at all different levels, so the service centers use 
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our stuff, the school districts use our stuff, higher ed uses it, and we ’re the support 
system for it. 

As the Texas SSI looks to the future, developing district-level commitment to reform continues 
to play a key role in how its leaders see the reform moving forward, with plans to “build a service 
for district and school board leadership.” 

Changing the Vision of Effective Professional Development 

As a result of all of the leadership development undertaken by the SSIs, one common outcome 
stood out. Across SSIs, leaders noted that administrators’ views of what constituted effective 
professional development for teachers of science and mathematics changed substantially. One 
leader summed up the prevailing view of professional development at the beginning of the SSI: 

Districts were still expecting the traditional model, or a “How to Succeed at [the state 
assessment] ” workshop. So when providers said, “ Let’s work with teachers over the 
year, one afternoon a month, examine student work, ” things like that, the districts would 
say, “Are you crazy ? Can 7 you just come for one and a half hours after school one 
day?” So the field is pretty unsophisticated. They know one way and that’s what they 
understand. They ’re confounded by a new vision for professional development. 

A number of SSI leaders saw that view changing in their states. Leaders in Arkansas, Michigan, 
Ohio, South Carolina, and Vermont all noted that, particularly in districts where the initiative has 
been most active, administrators’ preferences for professional development for teachers of 
mathematics and science has increasingly favored intensive, job-embedded, content-rich, and 
long-term approaches. For example: 

One of the more interesting findings has been that principals have changed their ideas 
on what is worthwhile professional development. Rather than sending the teachers to 
one-day workshops, they prefer to send them [to professioncd development ] no shorter 
than those two-week [sessions]. So that’s been a change in attitude that I found 
absolutely amazing. I never would have believed that would have happened. (Ohio SSI 
Principal Investigator) 

It’s a different environment now. There’s more sophistication in some districts about 
what they ’re looking for and what they want in professioncd development. (Michigan 
SSI leader) 

I think what the SSI did was it showed administrators that you didn ’tfix the problem with 
a one day math workshop. ...I think we educated them here in the SSI that’s what the 
research said about changing student improvement, that it’s a multi year process, that it 
has various components. I think that’s the thing that the SSI did that no report could 
ever reflect is that it educated administrators about what meaningful staff development 
would be. (Arkansas SSI leader) 

In some cases, the changing views of what constitutes effective professional development was 
evident at the state level as well, a key element in sustaining the reforms. As the agency through 
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which federal and state allocations for professional development typically flow to districts, and as 
the agency responsible for implementing education policy statewide, the view of professional 
development in Departments of Education plays a key role in many states. The Texas SSI took a 
very proactive approach to get the state to establish priorities for funding the kind of professional 
development the SSI advocated. Training existing and new professional development providers 
in colleges and universities “how to apply for a higher ed Eisenhower grant,” a PI said, “What we 
did was create enormous demand, because in state agencies, when there are discretionary dollars 
they tend to follow demand.” 

Similarly, although traditional, short-term professional development still exists in Ohio as in 
other states, the success of the SSI interventions, supported by student achievement gains, has 
altered the state agency’s view of effective professional development. Noting that the 
Department of Education is sponsoring Math and Science Academies that are a minimum of five 
days in length, one of the Pis concluded that “one major change in the state [is] that we’ve almost 
broken the myth that the two-day workshop works.” 

Growing Leadership to Sustain the Reform 

The designation Initiative turned out to be an appropriate one for the SSIs. In the five years of 
Phase I funding, and even in the states receiving Phase II funding, none could claim to have 
attained systemic reform statewide. Rather, the success stories of the SSIs are tales of initiators 
of reform that has continued; stories of failure typically hinge on good beginnings that ended 
when the funding ran out. Through a host of strategies, many of the SSIs claim a legacy of 
setting change in motion in their states toward a higher quality and more equitable mathematics 
and science education system. 

In many states, ensuring that the work of reform would continue meant that the SSI had to 
address two critical challenges in the area of leadership. First, the initiatives worked to develop 
leaders that could pursue the reform vision immediately and also to continue to do so long after 
the SSI funding ended. Second, the initiatives established an organizational base from which 
new leaders could work. 

A legacy of new leaders and a more coherent vision for mathematics and science education 
among existing leaders was commonly underscored across states. At the same time, those states 
that failed to leave a strong organizational base for leadership for systemic reform expressed 
doubts about the long-term viability of the reform. Two observers in Michigan offered candid 
thoughts about the future of systemic reform in their state: 

Systemic change takes a long time. Five years gets you started, then you have that long- 
term implementation process, with people wrapping their heads around it, and getting 
people connected, and having someone there to remind them, to organize them, to bring 
them together to talk and make plans, and reconvene them. No one ’s asking them to do 
that now. (SSI Project Coordinator) 

The other thing that’s missing today — the one thing the SSI did do and nobody’s doing it 
now — it was a way to convene people. If you don 't convene people, you can ’t keep a 
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common vision going. Now it only happens within an organization, and not at a higher 
level than that. The MDE has not taken that role. (SSI Evaluator) 

One leader saw a similar impact of the Nebraska SSI and had similar doubts about the future of 
reform: 



The legacy is really held within the people who had this experience. The legacy is not 
really held within the basic organizational changes. I’m afraid that with SSIs, that was 
the original intent — that organizations and the way of doing business would change so 
dramatically that the infrastructure would be there as people came and went. In reality, 

I think the legacy is held within the people rather than in a major lasting infrastructure. 

It has changed the way people do business and made them more connected, but when 
they leave, the legacy goes with them. The SSI changed the landscape in Nebraska. It 
created some connections and networking, but it’s by the good will and the experiences 
of the people who are in place. When they go, these things will be history. 

Seeing a need to provide an organizational base to continue the work of reform, the Arkansas SSI 
Project Director took measures to assure that at least some of the leaders the SSI developed 
would continue to operate in support of the reform the SSI had begun: 

At the end of the sixth year I went to our [Department of Education ] director. . . . And I 
said, “... I have an idea about how we could keep the SSI going. I want to write it up into 
our plan, but this plan, A, B, and C; we need 26 math specialists in the state. You could 
do just 15 now [for the Cooperatives] and add more of them later ... then 11 more for the 
university. One would be elementary and 10 secondary. ” We ’d do it in phases and I laid 
out what the scenarios were. “But the most expensive one, and the best one we ’ll put 
them cdl in at the same time. We ’ll train them all and get going here in our state. ” He 
loved it. 

... Once every six weeks or so I c cdl a directors’ meeting, the directors of cdl the centers. 
And we come together and share what we ’ve been doing in our different regions. We 
have two or three state projects a year that we work on together. They all have 
assignments. I go visit them. I guess it’s psychological; I still have the role of the 
director of SSI. People still say that. I just carry that on. That’ s kind of a perception 
that I just haven 7 cdlowed to let go on their part. 

A strategy that to date has been very successful in some states has been the continuation of an 
independent entity to provide the organizational base for leadership in reform. The SSIs in 
Connecticut, Vermont, and Maine have all worked with this strategy. The Texas SSI, although 
situated within a university, has operated out of a somewhat independent center that has 
established longevity in leading reform. It is worth noting that except for Maine, the states noted 
here all received Phase II funding so the long-term viability of these organizational bases has 
only begun to be tested beyond NSF support. Also, the Michigan SSI attempted to establish an 
independent organization as a sustaining home for reform in the state beyond NSF funding for 
the SSI, but the strategy has failed to live up to expectations. Still, the strategy appears to have 
worked in a number of places. 
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The Puerto Rico SSI, similar to Texas in its organizational base throughout NSF funding, has 
taken a somewhat different route with regard to providing an organizational basis for continued 
reform. A key factor in Puerto Rico was the leadership of the PI, whose importance to the reform 
was considered so great that leaders recall concern within the state and from NSF that the success 
of the SSI depended almost entirely on him. Developing broader leadership for reform was 
deemed a notable success of the initiative, as the PI and another leader said: 

NSF’s question early on made perfect sense, “What’s going to happen to the PRSSI if the 
plane that [ the PI] is on goes down ? Will the PRSSI go down too ? Or are there other 
people who can take over?” At the beginning of the project, we said, “Well. ..we’re 
working on that, ” And now we can say categorically yes. We have more experience, 
more exposure, and more mileage now. We have grown into our roles. And the way we 
play our roles and our strengths, we compliment each other. [The PI] allowed 
leadership to develop. There ’s good vision and good teamwork. (SSI Staff Member) 

If I dropped dead now, it would be fine. At the beginning, it would have never taken off. 
But we have built leaders now, others who have become good politicians. They 
understand very well what makes things happen. (SSI Principal Investigator) 

Looking at the long-term, however, the SSI does not intend to rely on its original organizational 
base to sustain reform. Rather, the districts and schools that the SSI has developed into Regional 
Professional Development Centers are expected to continue to provide the front line of reform, 
with the Puerto Rico Department of Education supporting them and providing coherence for 
reform across the Commonwealth. The PI expressed some reservations, similar to those 
expressed by leaders in other SSIs, that the reform might depend too much on the people the SSI 
developed as local leaders for the reform rather than on a changed system: 

The question comes down to this: Is there the political will? It shifts. It might be there 
for two years and then go away. The schools are empowered, and that’s a stabilizing 
element. But there ’s instability in the schools. The principal and teachers can shift 
around, and very soon the collective memory of what has happened is gone. 

Hope for the Department of Education to provide the organizational leadership needed to sustain 
the reform was high, however, as leaders in the SSI and the Department described: 

We ’ve identified what we want to be our legacies. We want to strengthen the leadership 
of the Centers — doing as much professional development for the Coordinators and 
teachers, and working with the Department of Education to [help them] understand all of 
this, to make decisions jointly about where we want to go — making sure that the materials 
are in shape, and ready for distribution, how the whole school strategy works, what the 
community of learners is all about, how we work on building alliances with community 
and industry, tools and products for the [Department] and others. The policies are there. 
I am hopeful. (SSI Staff Member) 
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You have to have a model that can be sustained within the budget. The PRSSI has always 
worked with more money than the system can afford. But the process of whittling it down 
might be good because it will make it more real— figuring out how to maintain reform 
within the reality of our budget. The Department supports the model. It’s a good 
exemplar. We can use it as a base for the work we want to do.... The principles and the 
work of the PRSSI are very good and we need to incorporate them into our programs and 
we ’re working in that direction. (Puerto Rico Department of Education Staff Member) 



Managing Interactions with the National Science Foundation 



“The work we’re doing here is pretty complex and you can’t get a good handle 
on it just reading reports. You’ve got to come on inside and interact with us 
and observe what we’re doing in order to give us constructive feedback.” (SSI 
Project Director) 



The SSIs were funded by an external agency, and therefore had another very important 
stakeholder to consider, in addition to those in the state, in maintaining support for their reforms. 
The fact that the funder was embarking on an initiative of unprecedented scale and scope added 
to the challenge. 

The National Science Foundation (NSF) has three primary funding arrangements. The first, and 
by far the most common, is a grant, where the Principal Investigator is given considerable leeway 
to pursue promising avenues; the primary constraint is that any modifications to the original plan 
be consistent with the objectives of the project described in the proposal, as amended during 
funding negotiations. The second arrangement is a contract, where the funder is “in charge,” and 
it is the Pi’s responsibility to carry out the instructions of the funding agency as described in 
detail in a contractual agreement. The third arrangement is a cooperative agreement. 

The SSIs were awarded as cooperative agreements, which are not as formal as contracts, but not 
as open as grants. Neither the NSF Program Officers nor the Pis of the SSIs were entirely sure 
about the parameters of this fairly unusual type of partnership as it began, or as it evolved over 
time. In addition, NSF was under pressure to show evidence that this major investment was in 
fact leading to improved mathematics and science education systems. The key challenges for 
SSIs in managing interactions with NSF were: 

• Developing a shared understanding of the strategy for the reform; 

• Negotiating appropriate changes to the design; and 

• Making a case that their initiative was having the desired impact. 

These challenges are discussed in the following sections. 

Developing a Shared Understanding of the Strategy for Reform 

As noted earlier, systemic reform was a new idea when the SSIs began their work. It was not 
immediately clear where the “system” they were supposed to reform was bounded, indeed 
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whether it was possible to change the mathematics and science education system without also 
changing it for all of the core subjects. In retrospect it is clear that a time frame of five years for 
changing even the mathematics/science education system was hopelessly optimistic. Said one of 
the Pis: 



I think at the beginning of the SSI project NSF didn ’t truly understand what systemic was 
and how difficult it would be to create systemic change. The fact of the matter is that 
when you think about doing a five-year state systemic initiative program — to create 
change in five years, that, in itself, I think was naive. ...So in some respects one can 
argue that a five-year systemic improvement is an oxymoron. 

Having taken on an unrealistically large task, the SSIs could not possibly do all they had 
promised to do, and consequently felt very vulnerable in their interactions with NSF. In 
interviews, a number of SSI leaders spoke about the frustration they felt in communicating with 
their Program Officers, who did not seem to have a deep understanding of the education systems 
in the various states. The following comments, from two different SSIs, were typical: 

It was a lot of pressure. Every time I talked to (the NSF Program Officer ) he kept saying, 
“ You ’re not a professional development project. You ’ve got to change policies. You ’ve 
got to work at the top level and change policies. ” Then I said, “Well, I know that, but in 
our state we have got to work at the grassroots level, and you ’ve got to work with the 
school board, and. . .you 've got to have a product that proves itself. ” 

At that time, we were just starting to provide a bit more state aid, but districts were very 
much locally funded and locally controlled. Things were pretty decentralized. There 
were no top down mandates about much of anything, which drove NSF crazy. They 
didn ’t understand that. They didn 7 understand why we wouldn 7 just charge in and tell 
local school districts how it’s going to be. That would have sunk the whole project and 
killed the key leadership. 

Leaders in another SSI talked about being confused by interactions with NSF; they felt like they 
were held up as an example of good systemic reform, but then did not get Phase II funding and 
were baffled. 

By all accounts, communication problems were exacerbated by changes in NSF Program 
Officers, forcing the SSIs to spend a lot of time and energy “educating the new Program Officer 
and bringing him or her up to speed on the project.” For example: 

Our problem with that was we never had enough stability in Program Officers for 
anybody to really get to know us very well. We went through about a Program Officer a 
year for a while. We had a couple of very good ones. They were getting to know us and 
about that time they rotated out or they left and went to something else. So that was a 
problem. . . . The work we ’re doing here is pretty complex and you can 7 get a good 
handle on it just reading reports. You ’ve got to come on inside and interact with us and 
obserx’e what we’re doing in order to give us constructive feedback. 
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We had this series of rotators. We had no one inside who knew what we were doing and 
was an advocate for us. 

In recognition of the scale and complexity of the SSI work, NSF took steps to create a 
multifaceted support structure for the program, comprised of three independent contracts. 
Through a technical assistance contract, SSIs would be able to meet together to share experiences 
and insights, and have access to experts in various aspects of reform. NSF Program Officers, 
who had very tight Congressional-mandated limitations on their travel, would be assisted by 
“monitors” who would visit the sites and in other ways keep a watchful eye on how each SSI was 
progressing. Finally, a national evaluation effort would look across the states for insights about 
the processes, problems, and impacts of statewide mathematics and science education systemic 
reform initiatives. 

A report on the Wingspread Conference held in March 1994 with a group of Pis, NSF Program 
Officers, and Contractor staff (Horizon Research, Inc. et al., 1994) noted general agreement 
among the Pis that a support structure was important, and appreciation that NSF had recognized 
this need. At the same time, the Pis expressed a great deal of frustration about some of the 
components of the support structure, and a general concern about “the lack of a systemic 
perspective throughout the initiative,” citing examples where the various contractor activities 
“aren’t as supportive of the overall effort as they could be.” 

Interviews conducted for this study indicate similar views roughly seven years later. Pis and 
other key leaders were able to recall specific instances of help, especially from the monitors who 
“came in and got to know us very well.” They continued to express disappointment about the 
utility of the technical assistance provided, noting that, “You’ve really got to get involved in it to 
be able to help anybody to do it; I’ve never seen the technical assistance providers involved in it 
enough to help.” And they continued to express frustration that the program evaluators came in 
to collect data, but that the results were not available in time, or in a form, to be useful to the SSIs 
themselves. 

It is interesting to note that at the same time they were experiencing difficulty in communicating 
with NSF and with the NSF contractors, a number of the SSIs were able to use NSF for 
considerable leverage in their work. Explained a key leader in one of the SSIs: 

NSF was always the silent partner, in that when you needed to do something or needed 
something to take place, you could always say, “NSF expects us to do this. ” Not that we 
shouldn ’t do it or didn ’t want to do it necessarily, but it was always nice to say, “If we ’re 
going to continue to take NSF money, then we need to do these things. ” 

The resources the SSI awards brought to the states were clearly helpful, especially in the smaller 
states. More importantly, the awards brought with them the prestige of the National Science 
Foundation. SSI leaders were able to use the clout of the award to successfully leverage internal 
and external funds to support pre-service reform, professional development, community 
outreach, and other activities. Said one of the Pis: 



Horizon Research, Inc. 



70 



March 2003 




I know this is going to break NSF’s heart, but quite frankly the (amount of money) that 
we were getting a year was pittance money. I mean it was so little that the Commissioner 
of Higher Education didn 7 pay much attention to it, just from the dollar point of view. 

He ’d pay a great deal of attention to it from the prestige point of view, and the 
partnership point of view. 

Negotiating Appropriate Changes to Improve the Design of the Initiative 

Whether at their own initiative, or in response to policy decisions higher in the Foundation, from 
time to time NSF Program Officers requested changes to the designs of the SSIs. In interviews, 
Pis expressed frustration with some of these requests, believing that NSF did not understand the 
likely consequences of their suggestions. For example, one SSI noted NSF’s concern that 
science did not receive as much attention as mathematics did, and the difficulty the Program 
Officer had in accepting the reasoning behind that decision. Explained the PI: 

It’s not that we didn 7 do work in science, but we certainly didn 7 put much emphasis [on 
it] because of the accountability system. We didn 7 have a leverage point early on to do 
science work and we did in mathematics, but at the same time we continued to work to 
create that leverage point and we now have it. We got it in at the last legislative session, 
because science is now a part of the assessment and accountability system. 

Other SSIs felt pressured by NSF to scale-up prematurely, beyond their capacity to do good 
work. For example, one SSI had been funded as a middle school initiative, but was soon asked to 
expand to target all of the mathematics and science teachers in the state, K-12. Said the PI, “It 
just wasn’t going to happen with $10 million in a state.” 

Another SSI hoped to establish a strong presence in about 200 schools in the first few years, 
steadily building capacity so they could expand to a much larger number of schools after that. 

But there was pressure from NSF over the small proportion of schools to be involved, and the 
scaling up strategy. With their knowledge of both the strengths and the challenges in the system 
they were seeking to change, SSI leaders were adamant about keeping the design, as 
demonstrated by this excerpt from the written response to NSF after site visitors expressed 
concerns: 

It must be considered that as the number of schools increases, the variability of contexts 
and the complexity of the project will increase. Therefore, to increase the number of 
schools beyond 210 would put the quality of the changes in jeopardy, as it would not 
allow for quality control, and the demands placed on [project] staff, without a 
substantial increase in resources, would not be manageable. The progression must be 
orderly and gradual. 

One of the leaders of this SSI recalled, “We risked losing support from NSF, but it would have 
been a greater loss to cave in and do a superficial approach.” In the end, NSF accepted the SSI’s 
reasoning, and the initiative continued its planned trajectory for scale-up. 

In other cases. Pis reported that NSF requests did in fact result in major improvements to their 
initiatives. For example, NSF was opposed to one SSI’s plan to develop an integrated 
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mathematics/science framework. The SSI staff leading the framework development 
subsequently tried to steer a course that maintained their commitment to much of the original 
vision while being sensitive to the concerns that were being voiced. In the final version, science 
and mathematics retained their separate identities, with standards derived from national standards 
documents. The interdisciplinary flavor was still present, but not in the foreground as originally 
intended. The revised framework was more acceptable, both to the broad mathematics and 
science education community and to NSF. Another SSI realized in retrospect that it should have 
heeded NSF concerns earlier about their spending too much of their time and energy on materials 
development given everything else they were trying to accomplish. 

Providing Information on the Quality and Impact of the Reform Interventions 

The federal government had invested considerable resources in the SSIs, and Congress 
understandably wanted to be sure the SSIs attained a worthwhile return on that investment. The 
SSIs were charged with changing systems, and although there was general recognition that doing 
so required gamering support, leveraging resources, aligning policy, increasing capacity, and 
developing infrastructure, it was far more difficult to measure these fairly abstract constructs than 
it was to count the number of teachers involved in the SSI interventions. Congress would almost 
certainly have preferred hard evidence of the impact of the SSIs on student achievement, so 
NSF’s plan to collect data on a more proximal measure — number of teachers reached — was in 
effect a buffer, buying the SSIs more time to fine-tune their interventions and to work to ensure 
high quality implementation in the classroom before being held accountable for student gains. At 
the same time, the need to provide evidence of numbers of teachers reached created an incentive 
simply to provide direct services rather than strategically devoting time, energy, and resources to 
the other parts of the system. 

As noted earlier, the SSIs realized early on that they could not do everything they had hoped to 
do, and proceeded to make choices, explicitly or by default. Some SSIs took the charge of 
changing the system very seriously, and put a great deal of effort into aligning policy in support 
of the reform vision. As a result, they had fewer resources available for providing direct services 
to teachers, or for assessing the impact of that work. The “attribution” issue — NSF’s emphasis 
on the impact of systemic reform activities on student achievement — was a particularly thorny 
one for the SSIs that focused their work on policy alignment or on developing capacity and 
infrastructure at the regional level, making it more difficult to determine the SSI’s influence at 
the classroom level. In the words of one of the monitors, “The numbers game didn’t work in 
[such states].” 

NSF’s introduction of the “Drivers of Systemic Reform” several years after the SSI program 
began was similarly well-intended, but it had unforeseen negative consequences. Noting the 
need for progress in areas such as aligning policy, leveraging resources, and engaging the 
community, as well as seeking improvement in student achievement, the drivers were meant to 
highlight for the SSIs that they needed to attend to the entire system. While broadening the 
vision of the charge helped broaden the view of SSIs that might have thought of their efforts as 
large professional development projects, the message that many SSIs seemed to take from the 
drivers was that they simply had to show some activity in each area each year. So, for example, 
SSIs described the number of Family Math/Science nights they had sponsored, or the number of 
pamphlets distributed to the general public, without describing a plausible plan for gamering 
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broad community support for the reform vision. Ironically, the very breadth of some of the 
drivers, such as public support, may have served to make the SSIs less strategic, distracting them 
from attending seriously to the parts of the system where they might have had a major influence, 
especially in the area of policies that could continue to provide guidance and incentives for 
ongoing improvements after the funded period. 

Pis noted that the introduction of the drivers, as well as other program requirements — the “core 
data” collection, mid-point reviews, and program effectiveness reviews — created a lot of 
problems for the SSIs. The following comments were typical: 

We had 614 districts in this state and we simply could not collect data the way NSF 
wanted it. We ’re a local control state. We had to depend on volunteers to give us data 
and little districts do not have those kinds of personnel. The drivers were very 
complicated for us. They made our life very difficult. 

[The drivers were] used as a hammer, maybe even a sledgehammer. It just seemed in the 
last year and a half that all we did was to write reports and they would surprise us again 
about data where we 'd say, “How the hell are we going to get this? ” It was not a 
community where we discussed what needed to be done. It was a community in which 
they told us what we were going to do. 

SSIs also spoke of the pressure they felt to demonstrate the impact of their interventions on 
student achievement well before one could reasonably expect impact, noting that they were not at 
all surprised when they could not find evidence of impact on students. Sometimes they found 
gains they themselves could not explain. Said one PI: 

It was a little bit ridiculous, but that ’s what they wanted. . . . Whether ( the improvement 
we found) was real is another question, but I will tell you we showed increases on ITBS 
fourth and eighth grade and whatever the equivalent eleventh grade test is. We did a 
comparison of schools that have been with us for three or more years, schools that have 
been with us one of the three years, and then comparison schools. Then we matched 
them demo graphically, free and reduced lunch, the whole bit, and we showed differences. 
What caused that, I can ’t tell you; nobody can. 

Project leaders in another SSI noted their plan to devote more effort to gauging student impact 
during the proposed Phase II: 

Up to this point, most of the evaluation has focused on teacher change, which is a critical 
process outcome. Without teacher change, little student change could be expected. As 
we learn that teachers are changing the way they think about, teach, and assess 
mathematics and science, we can turn to them to learn about how best to assess the 
changes in their students ’ achievement. 

But without a mechanism for collecting baseline data during Phase I, or for gauging the impact of 
the SSI activities on students whose teachers had participated in SSI activities, project leaders 
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were at a loss in trying to convince NSF of the impact of the reform. A PI in still another state 
described a similar problem: 

I think in the beginning we didn ’t think too much about assessment at all and we didn 7 
plan too much about how to gather data in the beginning. We were all neophytes. ...We 
had a lot of trouble the first two or three years trying to figure out where was the data, 
how do you get it, what does it mean, and still what we would present to NSF wasn 7 
enough. So that was the piece that by the end of the SSI we were realizing some things 
that we should have done early on. 

To complicate matters even more, in most cases there were other key players besides the SSI in 
mathematics and science education in the state. Said one PI, “We couldn’t always take direct 
credit for (impact) because we’re kind of infused in the process.” As noted earlier, assuming 
credit for improvements in this context was a delicate issue, risking rankling others engaged in 
reform efforts and rekindling turf issues. At least one SSI thought in retrospect that they erred on 
the side of not “bragging enough” about their accomplishments, noting that people from other 
SSIs were more open in taking credit for activities in their states. 

In summary, the high visibility of the SSI and the need for NSF to show returns on the federal 
investment, combined with frequent changes in Program Officers, led to what the Pis perceived 
as changing the rules in the middle of the game. The SSIs found they needed to devote 
considerable time and effort to managing their interactions with NSF, trying to ensure that their 
Program Officers (and the technical assistance, monitoring and program evaluation contractors) 
understood the SSI’s plan, could provide helpful feedback, and would not make requests that 
undermined the success of the initiative. 
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Part III: Summary of Lessons Learned 



This study of the Statewide Systemic Initiatives Program was undertaken to address the question: 
What lessons have been learned about designing, implementing, evaluating, and supporting 
statewide systemic reform? Our investigation centered on SSI leaders’ strategic planning, 
decision-making, and thinking about design, implementation, evaluation and support for systemic 
reform; and about the appropriate role of the SSI within the state education system. 

The research design for the study drew on SSI leaders as the primary data source, with 
documents as a companion data source. As such, the research was primarily qualitative in nature. 
The analysis design evolved with the research. As each round of data collection was completed, 
the study coordinators developed, used, and refined coding schemes; wrote interim products; and 
solicited reviews of interim products from SSI leaders, research partners, and advisors. 

The findings of the study are consistent with the focus of the research question — lessons learned. 
The leaders of the SSIs learned a great deal about statewide systemic reform as a part of the 
enactment of reform in their states through the SSI program. Many of these lessons learned were 
similar across SSIs, and both “positive” and “negative” examples supported the lessons learned. 
That is, leaders often came to similar conclusions about what was needed in statewide systemic 
reform in certain areas, pointing to the same factors as either important to success or the lack of 
those factors as barriers to success. 

In the analysis and reporting of lessons learned, we found it useful to address the technical 
strategies and demands, political strategies and demands, and interactions with funders 
separately, although we do not view them as independent by any means. A summary of lessons 
learned is presented in the same way here, with attention to the interdependencies of the lessons 
learned. We conclude with a design tool that we offer to the mathematics and science education 
community as a way to use the lessons learned in the SSI Program to analyze and improve the 
design, implementation, evaluation, and support of reform initiatives. 



Technical Strategies for Systemic Reform 

We have used the term technical strategy to describe how the SSIs thought about, and went about 
the work of, establishing the kinds of interventions needed to bring about changes in teaching and 
learning that result in improved and more equitable student achievement. The main aim of an 
initiative’s technical strategy, as we have defined it, was developing human and organizational 
capacities for reform. 

The view we developed in the study of a sound technical strategy was one that: 

• Operationalizes the reform vision through interventions; 

• Monitors and refines the interventions, and provides evidence that they result in 
improved classroom practice and improved student outcomes; 

• Increases capacity within the state to scale up the reform efforts; and 
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• Provides evidence that quality and impact are maintained during scale-up. 

If it is true that “actions speak louder than words,” then the interventions of a systemic initiative 
are what communicate what the initiative is really all about. Operationalizing the reform vision 
through interventions was a way to demonstrate a commitment to ideals of high quality 
mathematics and science teaching and learning, and to show how that commitment translated into 
a real effort to change the system. Two critical commitments that had to be communicated 
through the initiatives’ interventions were the central role of teachers and the need for equity. 

SSIs stressed the critical role of teachers through a large investment in professional development, 
attending to teachers’ knowledge and skills as necessary for substantive change in teaching and 
learning. SSIs demonstrated their commitment to equity by targeting early and middle grades 
where student achievement gaps begin to show up and students have not yet “opted out” of 
mathematics and science study; and by focusing their efforts where the greatest needs were 
manifest. 

The SSI Program was among the first in a long-term trend of increasing attention to internal and 
external research, evaluation, and monitoring in NSF’s education programs. In terms of technical 
strategies, SSI leaders learned to benefit from these efforts by paying careful attention to the 
quality and impacts of their interventions in order to refine and improve them throughout their 
implementation. They also learned to collect and use evidence of quality and impacts to 
demonstrate that the interventions produced the desired results. 

We noted three key lessons learned in this area. First, in designing the interventions, beginning 
with something with a manageable scope and scale could be very beneficial. Starting small 
helped ensure that the initial interventions could be implemented as planned, to work out the 
kinks in the intervention strategy, and to gather early evidence on how and to what extent the 
intervention produced the intended results. Second, although it was tempting to focus resources 
on scaling-up the interventions, it was more important to understand and establish data systems 
to monitor and conduct research on quality and impacts. In that way, SSIs could be sure before 
and during scaling-up that the interventions would be well-received and that they would work as 
intended. Third, the design of interventions had to be flexible enough to take advantage of what 
was learned through research, evaluation, and monitoring. Finding out what worked well and 
what did not resulted in substantial payoff when changes could be incorporated, but only 
produced frustration when the changes that were suggested could not be enacted. 

Our study reinforced the view that an “initiative” is a catalyst that sets a transformative change in 
motion, rather than a force that itself produces all desired changes. Even in small states, the SSIs 
did not begin with the capacity needed to reach all districts, schools, and classrooms in their 
states. A major pursuit in nearly all of the SSIs was to increase capacity to deliver the SSI’s 
interventions on a large scale. 

Three key lessons emerged in this area as well. First, many kinds of capacity were needed to 
initiate and continue reform. SSIs needed the human capacity to deliver their interventions, a 
considerable need in and of itself in many SSIs. SSIs also needed the organizational capacities to 
house and to support reform, and the leadership capacities to direct, to manage, and to monitor 
reform were not in existence in most states. Building these capacities became a focus for many 
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SSIs. Second, although scaling-up too quickly could be problematic, an initiative needed a 
feasible plan early on for scaling-up within a reasonable time frame. Without such a plan, it was 
difficult to judge or to justify whether and how the early efforts of the initiative could be 
expected to reform a large education system. Third, capacity building for scale-up and the need 
for quality control were in constant tension. Making this tension a healthy one was absolutely 
necessary, because ultimately the interventions and support for reform had to expand well 
beyond the initiative’s core players. And since scaling-up ineffective interventions would not 
benefit teachers or their students, a system for quality control during scale-up was needed. 

Few SSIs were able to provide clear evidence that intended quality or impacts were maintained 
during scale-up. Based on the few that attended to this area, however, it appears to provide an 
important lesson. Research, evaluation, and monitoring played a pivotal role in systemic reform 
in initiatives that collected information and provided evidence that the capacity building and 
quality control measures of the SSI could in fact deliver on the goals of reforming a statewide 
system. 



Political Strategies for Systemic Reform 

We have used the term political strategy to describe how the SSIs thought about, and went about 
the work of, establishing a supportive context for reform. The main aim of an initiative’s 
political strategy, as we have defined it, was fostering the professional, political, and public will 
for reform. 

The view we developed in the study of a sound political strategy was one that: 

• Facilitates development of formal policies that provide guidance and incentives for the 
reform vision; 

• Cultivates broad understanding of and support for the reform vision; and 

• Increases school and district leadership commitment to reform. 

One of the signal features of systemic reform is the focus on education policy in addition to 
service delivery. Education policies provide guidance and incentives for districts, schools, 
teachers, and others to pay attention to specific aspects of teaching and learning, and to take 
advantage of particular services. Coming at a time in the nation’s educational history when 
greater centralized policy guidance was emerging even in traditionally “local control” states, the 
SSIs entered state landscapes where education policy was a key factor. Although many SSI 
leaders had not been active players in shaping education policy, most took on this aspect of the 
work as a necessary part of their role in the SSIs. 

A number of important lessons emerged out of SSI experiences in the policy arena. First, 
housing the initiative within, or forming a partnership with, organizations that play key roles in 
developing and/or implementing policy positioned many SSIs to engage in policy work. Second, 
involving education policy makers as leaders or partners in the initiative often situated the SSI as 
a natural contributor to policy decisions affecting mathematics and science education. Third, 
providing an “existence proof’ of high quality, valued service and contribution in one policy area 
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often opened doors for the initiatives to play an expanded and ongoing role in their state’s 
education policy arena. In some cases, SSIs took the lead in developing position papers, 
frameworks, or reviews that became important resources for subsequent policy developments in 
their states. These products heralded the intention of the SSI to be player in the policy arena and 
demonstrated the capability of the SSI to do so. Third, either as a contributor to education 
policies or through monitoring of the policy environment, initiatives often had an inside track on 
what was being offered to, and required of, districts and schools. Some SSIs took particular 
advantage of that information to reposition the SSI continually to aid local administrators in their 
responsibilities. Consequently, the mathematics and science expertise and services and the 
systemic improvement perspective of the initiative were integrated with assistance that many 
local district and school administrators sought from the SSI. 

In order to cultivate broad understanding and support for their reform visions, it was important 
that the SSIs establish certain “presences” in their states. The simultaneous trends occurring in 
most states toward centralized policy guidance and decentralized decision-making to respond to 
policies meant that the initiatives needed a presence at both the state and local levels. Lessons 
learned in this area mirrored these needs. First, the initiatives needed to establish the 
understanding and support of state-level leaders in mathematics and science education and in 
education more broadly. By establishing relationships with these leaders, the initiatives 
increased the likelihood that they would be informed about and consulted on state education 
policies and programs. Second, the initiatives needed to foster understanding and support from 
regional and local leaders. In local-control states and increasingly in all states, being a 
“statewide” initiative was very different from being a “state” initiative. Reaching into schools 
and classrooms required the SSIs to nurture relationships throughout the state with those who 
translated state guidance and incentives into local service delivery, support, and actions. Third, 
parents and other local stakeholders had to understand, and more importantly favor, the kind of 
mathematics and science teaching and learning the initiative advocated. Some SSIs took an 
explicit grassroots approach to inform and involve local stakeholders; others relied on work with 
intermediaries to cultivate understanding and support locally. 

The charge of substantial and enduring change in mathematics and science education systems put 
the SSIs at political crossroads in their states. While seeking to transform mathematics and 
science teaching and learning, they could not afford to get so far “in front of the curve” of reform 
in their states that districts, schools, and teachers could not keep up with changes. Nor could they 
afford to ignore policies or constituencies that held sway in their states, even if they were not in 
alignment with the initiative’s goals. Finally, an initiative devoted to equity had to find ways to 
bring together the traditionally powerful and the traditionally underrepresented voices in 
mathematics and science education. Two particular lessons were learned in the SSIs around 
these realities. First, the initiatives benefited from establishing neutral political turf to bring 
constituencies together. Maintaining a connection to, but a reasonable distance from, existing 
agencies and important political trends allowed some initiatives to become conveners of a broad 
array of stakeholders. In this way, these initiatives built a reputation for facilitating collaborative 
solutions that served and satisfied statewide interests and multiple constituencies. Second, as 
collaborative ventures and as complements to other reform efforts in their states, the SSIs had to 
balance taking credit and sharing credit for progress and successes. It was important that the SSI 
be seen as valuable to mathematics and science education in the state, but it was equally 
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important that the partners of the SSI and other players in mathematics and science education be 
given credit for their contributions, too. 

One group of stakeholders that SSIs found to be more crucial than most originally anticipated 
was local district/school administrators. Many forces exist locally that have a profound influence 
on what happens in schools and classrooms. The SSIs worked in many different ways to 
cultivate the will of local leaders to support reform. First, an important role of principals and 
other school leaders is teacher evaluation. The need to educate these groups with a new vision of 
high quality mathematics and science instruction was a lesson many initiatives learned. Second, 
in many states local committees make materials adoption decisions, and instructional materials 
play an appreciable role in defining the content and pedagogy of classroom instruction. Working 
with adoption committees to establish criteria and processes for selecting high quality materials 
aligned with the reform vision was a productive activity when the opportunity arose. Third, local 
leaders also make many decisions about the nature and extent of professional development that 
will be offered or required of teachers. Many SSIs could claim successes in changing views of 
effective professional development among local administrators and decision-makers and saw 
these changes as representing a substantial and lasting accomplishment. Fourth, since the 
beginning of the SSIs, the responsibilities of local education administrators has changed, 
especially in the areas of data-informed decision making and accountability. Administrators are 
increasingly being required to collect, analyze, report, and use data in their decisions and actions. 
Many SSIs found themselves in strong positions to work with administrators to develop systems 
and capacities for these new responsibilities, and consequently to introduce a systemic view of 
management in many localities. 

Another important lesson learned in the SSIs was about the need and opportunity to grow new 
and expanded leadership for mathematics and science education reform. The long-term and 
large-scale nature of systemic reform raised the need for initiatives to develop new leaders who 
could carry the reform throughout the state and into the future. The SSIs presented a favorable 
breeding ground for leadership development and many states can trace current and budding 
leaders in mathematics and science education to the SSIs. 



Managing Interactions with Funders 

The SSI Program launched a series of NSF programs developed around a commitment to 
providing a challenging and meaningful science and mathematics education to all students 
through changes to whole systems of education. It also pioneered a new relationship between 
NSF and program awardees in the form of cooperative agreements. The experience of the SSIs 
provides some important lessons to guide these relationships in the future. 

The critical aspects we found in the study of how initiative leaders managed their interactions 
with NSF included: 

• Developing a shared understanding of the strategy for reform; 

• Negotiating appropriate changes to the design; and 

• Making a case that their initiative was having the desired impact. 
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The SSI solicitation invited and elicited a wide variety of approaches to systemic education 
reform, a concept that was still very much in development when the program began. Even with a 
more developed understanding of what is meant by systemic reform, however, it has become 
apparent that the design and implementation of systemic education reform is very context 
sensitive and dependent. As such, it is critical that the leaders have a clear understanding of their 
own reform strategies that can be shared with the funder. On the funder’s part, it is vital to 
develop a deep understanding of each initiative’s reform strategies, how they fit the local context, 
and how they are expected to change the targeted education system. 

Based on such a shared understanding, initiative leaders and funders can work to negotiate 
appropriate and necessary changes to the design and implementation of an initiative. In systemic 
reform, this need is particularly strong, as interventions may take some time to come to fruition 
in terms of measurable impact, scale-up, or sustainability. A shared appreciation of the trade-offs 
and balances of the reform strategies are critically important in this area. For example, changes 
to the design may adversely affect the intended balance between scale-up and quality control. 
Also, although it is tempting to scale up interventions as quickly as possible, it may be more 
strategic in the long-term to focus on smaller-scale interventions with strong research and 
evaluation components in order to gather evidence and make the case that the interventions are 
effective. 

Finally, the initiative leaders must develop and deliver on a plan to provide evidence of whether 
the initiative is having the desired impacts on the education system and its outcomes. In the 
current era of accountability in education, it is the responsibility of initiatives to report progress 
and impacts to meet the needs of funders, who in turn are expected to provide this information to 
their stakeholders. However, the evidence has to match the nature, extent, and timing of the 
initiative as well. Again, a shared understanding of the reform strategies, expected impacts over 
time, and long-term outcomes of the initiative can guide appropriate collection, reporting, and 
interpretation of evidence that should benefit both the funder and the systemic initiative with 
useful information. A critically important corollary of this lesson is the need to establish 
appropriate, but flexible, indicators, data systems, and research and evaluation designs early in 
the life of any systemic initiative. 



Conclusion 

The SSI Program has provided more than a decade of support for statewide systemic reform. 

The successes, struggles, and learning of the wide variety of parties involved in the efforts have 
much to offer the nation. We hope that the findings and insights presented in this report offer a 
useful account of the SSI Program and provide helpful guidance as the nation moves forward in 
its commitment to improving and transforming science and mathematics education systems and 
providing high quality teaching and learning for all. 

We believe the findings and insights of this study can be useful to those who lead, evaluate, 
provide assistance to, oversee, and make decisions about mathematics and science reform efforts 
today and in the future. We offer to these groups a set of questions that can be asked of the 
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design, implementation, and evaluation of a reform initiative. We believe that attending to these 
questions is a way to apply the lessons learned in the SSI Program to the benefit of future reform 
efforts. 
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Recommended Questions to Ask of 
Science and Mathematics Reform Initiatives 

Technical Strategies 

1. Are the critical aspects of the reform vision (e.g., equity, deep content learning, 
teaching for understanding) evident in the interventions? 

2. Do the interventions signal audiences to the depth of intended outcomes and the level 
of effort and commitment needed to achieve the reform vision? 

3. Are systems being built to gather evidence about the quality of the interventions and 
their impacts, and to use that information to improve the interventions? 

4. Is there a feasible plan for scale-up, including establishing both the capacities and 
resources for doing so? 

5. How will quality control be assured during scale-up? 

Political Strategies 

1. Is the initiative positioned to work on formal policies through its leaders, organizational 
arrangements, and partnerships? 

2. Is the initiative positioned as a convener of stakeholders with differing interests and 
viewpoints, to generate productive conversation and decision-making? 

3. Does the initiative have mechanisms for informing, involving, and seeking support 
from key stakeholders, including leaders in mathematics, science, and education; 
regional and local decision makers; and parents and communities? 

4. How will the initiative address the important roles of district and school leaders (e.g., 
teacher evaluation, materials adoption, professional development planning)? 

5. How will the initiative provide assistance to district and school leaders to meet their 
obligations (e.g., strategic planning, accountability reporting)? 

6. How will the initiative address the need for developing new leaders for broadened and 
ongoing reform? 

Keeping the Funders Informed 

1. What information will the initiative provide to enable the funders to understand the 
context of the reforms, and the relationship between that context and the strategies of 
the reforms? 

2. Are the data systems in place, or being built, to provide evidence the funders will need 
of the quality and impact of the initiative? 
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Chronology Codes 



Pre - SSI Phase I 
Yr 1-2.5 of Phase I 
Yr 2.5-5 of Phase I 
Post - SSI or Phase II 



Component, Context, and Impact Codes 



A. INFRASTRUCTURE 

1. State structure 

a. Who were the leaders? 

b. What was state’s vision for reform? 

c. Relevant state context 

2. Reform prior to SSI 

a. What kinds of reforms were underway? 

b. Where were they housed? 

c. Who led them and where did the leaders work? 

3. SSI 

a. What was the SSI vision? 

b. Was it shared or fragmented? 

c. Where was the SSI housed? 

d. Who led the SSI and where did the leaders work? 

e. What were the SSI efforts to expand the SSI infrastructure (i.e. coalition 

building, bringing together different groups, etc.)? 

B. REFORM SUPPORT 

1. How supportive/ knowledgeable were key state leaders of standards-based reform 

in the state? 

2. What legislative efforts supported reform? 

3. Who else was supporting reform? 

4. Who else was opposing reform? 

5. What were the SSI efforts to build support/avoid opposition? 

C. STATE POLICY 

1. State assessment 

2. State standards/frameworks (alignment with national standards ) 

3. Accountability system (clear rewards/sanctions) 

4. Requirements for teacher preparation 

5. Requirements for teacher re-certification 

6. Requirements for graduation 

7. Text/materials adoption 

8. Other related education policies 

9. What were the SSI efforts to influence state policies? 



i 




D. SSI DIRECT INTERVENTIONS 

1. Description of Intervention 1 

2. Who was targeted? 

3. Evidence of quality 

4. Evidence of impact 

5. Numbers reached 

6. Scale-up strategies, effectiveness 

1. Description of Intervention 2 
etc. 

E. FISCAL RESOURCES 

1. Existing resources in state for mathematics and science education 

2. What were the SSI efforts to leverage resources? 

F. OUTCOMES 

1. How many people/schools/districts were reached? Who or which ones? 

2. Evidence of improvements in curriculum 

3. Evidence of improvements in instruction 

4. Evidence of broad-based improvements in assessments 

5. Evidence of improvements in achievement 

6. Evidence of increased equity 
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Appendix D 



Coding Scheme: 

Strategic Planning, Decision-Making, and Thinking 




Coding Scheme: 

Chronology and Components 

1. How did the leaders of the SSI read/assess the mathematics/science/technology system in the 
state in terms of . . . 

a. understanding/identifying the system's needs? 

b. understanding/identifying system's existing capacities for reform? (including 
standards/frameworks, assessments, and other guidance and support policies; 
communication/delivery systems/infrastructure for reform; expertise; existing or past 
proj ect s/programs ) 

c. understanding the system's operations and workings? (including important people, 
organizations, and system components) 

2. What did the SSI leaders consider to be ... 

a. the priority needs in the state (which ones and with what rationale?); 

b. the equity needs in the state (which groups were targeted by the SSI?); 

c. the proper/optimal utilization of existing capacities for reform in the state? 
(complementing, linking with, expanding, etc.) 

3. Describe the technical strategizing of the SSI in terms of how the SSI was envisioned to 
create systemic reform in the state including . . . 

a. building the needed capacity and will of teachers; 

b. building the necessary infrastructure for delivering and supporting the reform; 

c. gaining public/professional/political support and avoiding opposition; 

d. creating a supportive policy environment; 

e. catalyzing reform by putting the necessary and appropriate policies/structures in place 
during the funded period; and 

f. creating interventions targeting equity. 

4. Describe the implementation strategizing of the SSI in terms of how the SSI . . . 

a. provided incentives for organizations, groups, and/or individuals to participate; 

b. sequenced activities (with what rationale?); 

c. created existence proofs/proofs of concept to build momentum, gain support, etc. 

d. worked in areas providing widespread guidance, powerful influence, or high yield; 

e. maintained quality control throughout implementation; 

f. built in flexibility to adjust for barriers/opportunities; and 

g. made trade-offs regarding what was and was not addressed (with what rationale?). 

5. Describe the political strategizing of the SSI in terms of how the SSI . . . 

a. involved important and influential individuals, groups, and organizations; 

b. positioned itself to become the "voice" or umbrella for 
mathematics/science/technology education reform; 

c. communicated with key stakeholders, in terms of both dissemination and input. 
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6. Describe how the SSI monitored and evaluated its progress and impacts, including . . . 

a. collecting data on implementation, quality, equity, impact; 

b. anticipating/monitoring for barriers and opportunities; 

c. using monitoring or evaluative information for decision-making/adjustments/mid- 
course corrections; and 

d. disseminating evaluative information for publicity or other purposes. 

7. In the long-term, how did the SSI leaders plan to . . . 

a. take the reforms to scale in the state; and, 

b. sustain the reform (leadership, finances, self-renewal)? 

8. (Not for Phase I proposals) What SSI impacts and outcomes are described? 
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Appendix E 



Coding Scheme: 
Key Challenges 




Key Challenges for Statewide Systemic Reform 



The SSIs varied in terms of their “starting points,” each with its own demographics, reform history, and 
policy environment, and consequently the inherent set of needs and opportunities. Our task is to describe 
the extent to which each of the challenges we have identified was addressed by the SSIs (and the 
consequences, if any, of not addressing it); and how strategic their approaches were in their particular 
contexts. 

A. Creating and Supporting a Shared Vision for Reform 

1. Building a shared vision within the mathematics/science education community of excellence and 
equity in teaching and learning; 

2. Assessing needs and opportunities in relation to the vision; 

3. Establishing a recognized “voice” for mathematics/science education in the state; and 

4. Cultivating understanding of, and commitment to, the vision on the part of all key stakeholders, 
including key “power brokers” within the state, as well as parents and the broader community. 

B. Designing, Implementing, and Evaluating Reform Interventions 

1. Identifying/adapting/designing/sequencing interventions to “add value” to the state in support of 
the vision; 

a. Deciding when and how to link with existing interventions; 

b. Deciding when and how to expand to other areas of the mathematics/science education 
system (e.g., secondary, pre-service) 

c. Deciding when and how to link with or expand to other parts of the education system (e.g., 
literacy) 

2. Developing a plan for going to scale; 

a. Demonstrating that the interventions can be successfully implemented in this state context; 

b. Developing/identifying human capacity to carry out the interventions on a large scale; 

c. Developing/identifying infrastructure to carry out the interventions on a large scale; 

d. Securing resources to carry out the interventions on a large scale; 

3. Implementing the interventions; 

4. Monitoring the quality of implementation and outcomes of the interventions in order to make 
informed "mid-course corrections”; and 

5. Evaluating the quality/impact of the interventions in order to "make the case" for the initiative to 
external audiences. 

C. Changing the System so that the Reforms Become Institutionalized 

1. Fostering a culture in support of the vision throughout the formal education system (e.g., belief 
in the importance of elementary science education, understanding that on-going professional 
development is essential); 

2. Developing the leadership capacity, infrastructure, and partnerships to sustain processes of 
continuous improvement toward excellent and equitable mathematics and science education for 
all; 

3. Gaining access to system/policy levers to align curriculum, assessment, professional 
development and related policies in support of the reform vision; and 
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4. Supporting teachers, administrators, schools, and districts with the necessary resources, including 
high quality instructional materials, equipment, and supplies, as well as time for learning and 
reflection. 

D. Managing Interactions with Funders 

1. Developing a shared understanding of the strategy for reform; 

2. Using the funders for leverage within the state; 

3. Providing information on the quality and impact of the reform interventions; and 

4. Negotiating appropriate changes to improve the design of the initiative. 



2 




Appendix F 



Interview Protocol 

for Telephone Interviews and Site Visits 




Interview Protocol: 

Strategic Planning and Thinking in Statewide Systemic Initiatives 



In order to set the tone for what we expect to get from interviews, we need to 
communicate to interviewees: 

A. We have read background documents in order to get the basic SSI story and chronology. We 
are interested in their experience of the SSI, especially critical junctures in the development of 
the SSI and the "hows" and "whys" of the planning and management of the SSL 

B. We are aware of the enormous complexity of education systems and systemic reform and the 
limited resources they had to undertake the SSL We want them to be at ease to tell us what 
worked, what did not, what is still working or not, and why. 



I would like to begin with questions to help me understand the mathematics and science 
education system as it existed prior to the SSI, around (1990/91/92). It is important for us to 
understand the context that influenced the original SSI design. 

1. How would you describe the mathematics and science education system that existed at the 
time the original SSI plan was developed? 

Probe: What policies and activities were most influential in mathematics and science education 
in [state] in the years just prior to the SSI? 

Probe: Who were the leaders in mathematics and science education in [state] before the SSI was 
funded? What were their roles in mathematics and science education? Who, of these, was 
involved in the development of the SSI originally? In what roles? 

Probe: Were there influential individuals/groups who were not included in the SSI's "inner 
circle?" To what extent did this situation create opposition for the SSI? How did the SSI address 
this opposition? 

Probe: What else was going on in mathematics and science education at the time that the SSI was 
originally funded? To what extent were these activities anticipated to operate in opposition, in 
parallel, or in collaboration with the SSI? 
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Now, I would like to talk some about the original SSI plan — prior to funding and 
implementation. We hope to understand the original SSI design and the rationale behind it. 



2a. In reading the proposal, it appears the SSI plan focused on . Why did you 

focus on these particular areas? 

Probe: What trade-offs or compromises influenced the decision-making that led to the original 
SSI plan? 

Probe: What individuals/groups outside the group that developed the SSI proposal had input in to 
these decisions? 

Probe: Was the original plan altered in any important ways in response to negotiations with the 
National Science Foundation before the SSI was funded? In what ways was the plan altered? 

Probe: Were there any critical areas of the mathematics/science education system that the SSI 
plan intentionally did not focus on? Why were these areas given light treatment? 

2b. What were the critical and defining features of the original SSI plan? 

Probe: Which SSI activities or efforts were aimed at each targeted area for change? 

Probe: What were the most important implementation strategies for SSI activities? Why did you 
choose these strategies? 

Probe: What sequence of activities did the SSI plan specify? What was the thinking behind 
implementing the SSI in this sequence? [Keep track of short-term and long-term objectives.] 

Probe: Who was responsible for carrying out each SSI activity? How did you plan to prepare 
them for their roles? 
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Before we get into how the SSI unfolded, I would like to spend a few minutes discussing the 
SSI leaders' expectations for the initiative and the changes they anticipated would result. 

3. Given the initial status of the mathematics/science education system and the interventions 
planned for the SSI, in what ways did you expect the mathematics/science education system 
would be different at the end of the 5 year Statewide Systemic Initiative? 

Probe: What changes in the mathematics/science education system would have represented the 
success of the original SSI plan? 

[Be sure to attend to policy, support/opposition, infrastructure, teacher capacity and will, as well 
as a vision of teaching and learning.] 



Now, I would like to change gears and discuss how the SSI plan unfolded, focusing 
specifically on aspects of the original SSI plan that emerged as particularly important, as 
well as any changes in the direction or focus of the SSI. 

4. How did the original SSI plan evolve over the [first] 5 years of the initiative? 

Probe: What were your early successes? Were these in areas where you expected to show early 
results or were you surprised at these successes? 

Probe: Besides the SSI leadership, who else was kept informed about the SSI plans, progress, 
and results? How? 

Probe: Did you have sufficient resources to do what you had planned? In terms of money? In 
terms of people? 

Probe: What obstacles did you encounter that you had expected to encounter? Others you hadn't 
anticipated? 

Probe: What were the major changes in the SSI plan over the [first] 5 years of the initiative — in 
vision? in direction? in strategy? What prompted these changes? 

Probe (if necessary): To what extent did forces within the state determine the course of the 
SSI? 

Probe (if necessary): To what extent did NSF determine the course of the SSI? 

Probe: How did you know which things needed to change? What factors were critical in 
decision-making about how and when to alter the original plan? Who made decisions about how 
and when to alter the SSI plan? 
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Phase I only states 



Finally, I would like to talk about what the SSI accomplished. I am especially interested in 
knowing whether those accomplishments remain in evidence today and if so, how they have 
been maintained, or, if not, why not. 

5. What were the major accomplishments of the SSI in its [first] 5 years of existence? 

To what extent did these accomplishments meet the goals of the SSI? To what extent have 
these accomplishments been sustained/expanded since the original NSF funding ended? 

Probe/restate: How is the system of mathematics and science education different today than it 
was when the SSI started? 

[Ask about curriculum policies, assessment/accountability systems, teacher certification, 
professional development policies, graduation requirements, other policy areas already 
mentioned in the interview.] 

Probe: To what extent are those differences related to the SSI? To what extent are those changes 
likely to persist or continue? 

Probe: What are the most important forces in mathematics and science education in [state] 
today? To what extent are those forces related to what the SSI accomplished? 

Probe: What did the SSI do to assure that its accomplishments and its work would continue 
beyond the life of the NSF funding? To what extent were those efforts successful? 

Probe: Are any of the efforts initiated by the SSI still underway? How extensive are they today? 

Probe: What are the leaders of the SSI doing today? How are their roles in the mathematics and 
science education system today related to their involvement with the SSI? 

Probe: How did the SSI contribute to the emergence of new leaders in mathematics and science 
education in your state? What roles are these leaders playing today? 
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Phase II states 



Finally, I would like to talk about what the SSI has accomplished thus far. 

5. What have been the major accomplishments of the SSI? 

To what extent have these accomplishments met the goals of the SSI? 

Probe/restate: How is the system of mathematics and science education different today than it 
was when the SSI started? 

[Ask about curriculum policies, assessment/accountability systems, teacher certification, 
professional development policies, graduation requirements, other policy areas already 
mentioned in the interview.] 

Probe: To what extent are those differences related to the SSI? To what extent are those changes 
likely to persist or continue? 

Probe: What is the SSI doing to assure that its accomplishments and its work will continue 
beyond the life of the NSF funding? 
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Appendix G 



Case Report 

Arkansas Statewide Systemic Initiative 




Introduction 



The Arkansas Statewide Systemic Initiative (AR SSI) began in 1993, with funding from the 
National Science Foundation (NSF), totaling $10 million over five years. Noting the importance 
of high quality mathematics and science education for economic development as well as for 
personal fulfillment, the AR SSI set out to “restructure mathematics and science education by 
changing attitudes toward mathematics and science, improving student and worker skills and 
fostering long-term community involvement in the education system.” 

The strategy for achieving these ambitious goals was multi-faceted. Regional partnerships 
would be established, each comprised of schools, colleges, businesses, parents, and other 
community leaders. Professional development would be provided on a large scale; and 
classrooms would be equipped with innovative curricula, materials for hands-on activities, and 
appropriate technology. Cadres of mathematics and science leaders would be trained and 
supported in their work at the regional and local levels. Professional leadership activities would 
bring scientists and mathematicians into classrooms, and grade K-12 teachers into real-world 
research settings. Finally, by working toward changes in teacher preparation and certification, 
and by involving key policymakers in the statewide Steering Council and regional partnerships, 
the AR SSI would be able to “effect lasting change.” 

NSF intended the SSIs to be catalytic, not only to provide high quality professional development 
and other support to teachers, schools, and districts, but also to create an infrastructure and the 
momentum needed for the reforms to continue. Now, several years after the end of the NSF- 
funded period, this case study takes a look back at how the AR SSI went about its work, the 
barriers it encountered, the successes it achieved, and how the mathematics and science 
education system has changed as a result of these efforts. 

The documents reviewed for this report include the original proposal submitted to NSF, and the 
responses of the AR SSI to questions and concerns posed by the reviewers of that proposal; a 
monitoring report prepared for NSF by Abt Associates, covering the period 1993-1995; a case 
study by SRI as part of the overall program evaluation, covering the period 1993-1997; the 
summative evaluation report by the external evaluator. Accountability & Development 
Associates; the AR SSI Phase I final report; and the proposal submitted to NSF for Phase II 
funding. Seven interviews were conducted during a site visit in October 2002; another key 
player who is no longer in the state was interviewed by telephone. Their first hand accounts of 
the process, their pride in all that the AR SSI accomplished, and their disappointment that it 
wasn’t able to do all that they had hoped, shed considerable light on the realities of undertaking 
systemic reform. 
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The Context for Systemic Change in Mathematics and Science Education: 

A System on the Move 



Arkansas is predominately rural and predominately poor, lagging behind most other states in 
both the “quantity” of education (percent of adults with high school diplomas, percent with 
college degrees) and the quality of education; for example, on tests administered in the early 
1990s, Arkansas students scored well below the national averages on both the National 
Assessment of Educational Progress and on the ACT. 

Although Arkansas public schools have traditionally had a great deal of local autonomy, since 
the early 1980’ s there has been considerable state-level attention to improving education. 
Concerned about the poor quality of education in the state, and the resulting difficulty in 
attracting business and industry, then Governor Clinton appointed a commission to identify 
actions that could be taken to reform the education system. The commission’s recommendations 
led to the Arkansas School Standards Act of 1983, which required teachers to pass a competency 
test; established minimum performance levels for schools; called for greater attention to basic 
skills as well as more rigorous course offerings; and increased the requirements for high school 
graduation. 

This unprecedented attention to standards and accountability resulted in major changes in how 
the state went about the business of education, and sent shock waves through the system. As a 
staff person in the Arkansas Department of Education described it: 

[The 1983 legislation] was the first time that we had ever really begun to say what it was 
at the state level that students ought to know and be able to do. Subsequent to that 
legislation, we put in the first minimum performance testing program ... [at grades 3, 6, 
and 8] where 8th grade was designed at that time to be a cut score; you either passed or 
you didn 7 go on to high school. . . . There was a lot of resistance, because that was the 
first time when teachers began to sense that somebody besides the classroom teacher had 
some sense of direction of what they ought to be doing in the classroom. 

Despite controversy over the teacher competency test, and teacher resistance to being told what 
to do, the state maintained its commitment to standards, and the system began to change. 
Enrollment in advanced courses increased greatly, as did the high school graduation and college 
enrollment rates. Encouraged by the progress, but recognizing that the state still had a long way 
to go in improving its education system, in his second term, Governor Clinton pushed for 
additional reforms, resulting in the passage of Act 236 in 1991. 

Act 236 called for major overhauls to the system, requiring the Arkansas Department of 
Education to define specific learning goals, and to develop a plan to align statewide curriculum 
frameworks, assessments, and professional development programs with a vision that went well 
beyond basic skills. The legislation also called for the establishment of an Academy for 
Leadership Training to help build leadership capacity throughout the state, and a restructuring of 
the Department of Education to better enable it to provide technical assistance and support to the 
schools in their reform efforts. Also enacted in 1991 as part of the push for education reform 
were: an Educational Trust Fund, with a one-half cent sales tax increase to be used solely for 
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education improvement; the Arkansas School for Mathematics and Science, a residential 
program for students in grades 1 1 and 12; and a matching grant program to assist school districts 
in purchasing equipment for their mathematics and science programs. 



Seeking Support for an SSI: 
The Third Time’s the Charm 



One More Time? 

The original NSF solicitation for the Statewide Systemic Initiatives reached Arkansas during this 
period of intense attention to education improvement, and provided an opportunity for the state 
to focus more effort on mathematics and science education reform within the context of this 
overall reform. In the first two rounds, proposals were submitted by a technology office that 
was separate from both the Arkansas Department of Education (ADE) and the Arkansas 
Department of Higher Education (ADHE). Although several of the people interviewed for this 
report were at least nominally involved in those efforts, they did not have a great investment in 
them, and were hard pressed to remember what had been proposed. What they did recollect 
clearly was the general feeling in the third round that it wasn’t worth the effort to try again. 

Enter Diane Gilleland, the Director of the Arkansas Department of Higher Education. She and 
others in the agency were troubled by the fact that more than half of the students who graduated 
from Arkansas schools needed remedial mathematics at the college level, and they recognized 
that the higher education system was at least partly responsible. Said one of the key players: 

At that initial meeting, Diane kept saying “We have a problem with mathematics in this 
state. You can look at the number of remedial courses that are being offered at the 
universities. You can look at all the test scores that come in our K-12 system. As the 
director of higher education, 1 and my colleagues perpetuate the problem. We are the 
ones who educate the cadre of teachers out there, and we can 7 point fingers. ” 

Interviewees noted that low expectations seemed to have become an integral part of the system; 
many people took it for granted that only “some” students could do challenging mathematics. 
Reflecting back on the time when Arkansas was considering responding to the NSF solicitation 
for the third time, a member of the planning team recalled: 

There was still a very pronounced philosophy among many educators here that there was 
a group of kids that could, a group of kids that could do a little bit, and there was a group 
of kids that couldn 7 do, and that was okay. 

Finally, ADHE was already involved in a major professional development effort, a “Mathematics 
Crusade,” supported through a combination of state and local school district funds, Eisenhower 
grants, and private funds. During the period from the late summer of 1991 through the end of 
1992, the program had provided in-depth, extended professional development to more than 500 
teachers. The Crusades professional development, and the associated sets of calculators, geo 
boards, and other manipulatives to help teachers apply what they had learned in the classroom, 
had been very well received. 
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A New Plan 

The combination of data showing an important statewide need, concern about low levels of 
expectations for students, and the existence of a well-received model of professional 
development, prompted the ADHE to seek permission to submit a proposal for the third round of 
the SSI. Recalls the former Deputy Director of ADHE: 

The history was that the solicitation went out. It was the summer of 1992. Bill Clinton 
was running for President. We had in effect an Acting Governor and a chief of staff who 
mediated between the Governor and the Acting Governor. Diane Gilleland and 1 went to 
this person and said, “We’d like to give it another try. ” We knew that we had a base for 
it, which was this Crusade thing. And we knew we had to do it in partnership with the K- 
12 agency. ... And the other crucial thing was that the only matching money that was 
going to be brought to the table was from the Department of Higher Education, so we 
wanted to house the project there. . . . The Governor agreed to do that. 

Gilleland wanted to be sure that ADHE wouldn’t be stepping on anyone’s toes if they went 
ahead with a proposal. As another key observer recalls: 

She asked the other people in the Council, which was the director of education, the 
director of the workforce education and severed others, was anybody going to write for 
the third time because we could write one more time for the funds. Was anybody going to 
do it? Everybody scad, “No we’ve been turned down twice and no, we’re not going to do 
it. ” So she said, “Do you mind if the Department of Higher Ed makes a try at this?” 
Everybody said, “Yes, go cdiead and go for it. ” She checked again with the Department 
of Ed to make sure they weren 7 going to do anything. They said, “No, have at it. ” 

Interviewees seemed a bit surprised at questions about whether other options had been 
considered, and whether there was disagreement about which path to take. In contrast to some of 
the other SSIs, where there were a number of alternatives discussed, and competition among 
existing initiatives to decide which would be highlighted and expanded, the Arkansas planning 
team quickly settled on the Mathematics Crusade as the centerpiece of the AR SSI proposal. 

Said one member of the planning team: 

1 think we had edready moved in terms of doing some things in mathematics and had the 
Math Crusade there and it was beginning to get a toehold. 1 don 7 think there was any 
question [that this] is the direction that we need to go in. And the fact that we sort of had 
this thing put together, 1 think gave us the impetus to go ahead and bring science along in 
a like fashion. 

One of the people involved in the design of the AR SSI recalls being surprised at how quickly 
the decision was made that college courses were the solution, which in retrospect seemed to her 
to be an indication that people weren’t thinking about how to change the entire education system. 

1 thought one of the amazing things [was that] the immediate response from us was that 
we need to develop a course. 1 mean, it’s edways been the itch that college people, well 
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let’s get the syllabus out there, and if we can put the syllabus out there we can cure the 
problem. ... You know when I look back and I think about the times, I'm not sure how 
deeply everyone bought into the idea of system change. And I think as dedicated as 
everybody has been through this, it was still a sense of changing somebody or something 
else and it was hard to focus that it was us. It is the department of education. It is the 
department of higher education. It is the communication within each university, between 
their colleges of education and colleges of arts and science. 

Decisions about project management were made as readily as the decision to base the AR SSI on 
college courses, and were similarly lacking in controversy. Although the proposal development 
team included representatives from the ADE as well as other stakeholders, the plan was that 
ADHE staff would take the lead in project implementation. By mutual agreement, ADHE would 
serve as fiscal agent for the initiative, avoiding the bureaucracy involved in getting things done 
in the larger ADE. A staff member in ADE explained why housing the AR SSI at the ADHE 
was the right decision: 

That’s a very small agency up there and things can happen like that, (snaps fingers) You 
know here we’re 350 people, and this is one of a multitude of things that’s going on up 
here. Even though it was a priority for some of us, there were others in the food chain, 
that this was not the top priority. ... It’s where it had to be to keep it out of the politics of 
this building. 



The Design of the AR SSI: 

Expanding on a Pilot 

The proposal for the AR SSI described a vision of Arkansas as economically vibrant, with a 
highly informed and engaged public. 

Our vision is of Arkansas as fully competitive in the national and global economies. Its 
principal industries will produce high value-added products and services, thanks to the 
skills and motivation of workers educated in the schools and colleges of our states. We 
also envision Arkansas as a model of effective citizenship for all of our people. Through 
the quality of mathematics and science education in the state, our citizens will have the 
knowledge and the willingness to make judgments about complex issues in the workplace 
and in society. Their education and training will empower them to learn about and be 
interested in the key issues of a democratic society. 

The proposal also made it clear that Arkansas was very far from that vision. Among the 
problems cited in the proposal were the “dramatic drops in enrollments in mathematics and 
science classes after Algebra I,” and the particular need for “greater proportions of women and 
minorities (to) complete rigorous high school math and science courses.” 

In battling this back and forth I think what our philosophy became is you ’ve got to have 
someplace to start. ... [Regardless of the current levels of performance,] if there is 
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leadership support to provide these services, if they’ll participate, then we need to serve 
them. 

Systemic changes in Arkansas mathematics and science education were seen as the “chief 
catalysts to bring this vision to reality, [with] long-term efforts to change attitudes toward math 
and science education, improve skill levels in these subjects and foster community-based 
partnerships in every corner of the state.” Rather than targeting their efforts to a particular 
group, the AR SSI planned to “make it possible for all students to benefit from quality 
mathematics and science instruction... on the premise that every student can learn and achieve.” 

Improving Teacher Capacity 

The key element of reform, project leaders believed, was improving teacher capacity, although 
they also believed that high quality instructional materials were important. Explained one of the 
members of the planning team: 

You can ’t solve a problem by buying materials. You put bad teachers with poor 
preparation and good materials, you ’re going to get bad results. You put excellent 
teachers ...with bad materials and you’re still probably going to get good results. You 
put excellent or good teachers ...with good materials and there’s no telling what you can 
get out of the kids. 

Once they decided to focus efforts on professional development, and to build the AR SSI around 
the Math Crusades model, the planning team had to decide how to configure the initiative. Their 
goal was to provide the courses to large numbers of teachers, not just in mathematics in grades 
5-12, but in both mathematics and science in all grades. There was some discussion of the 
possibility of having one mathematics course and one science course, each serving teachers 
across the entire K-12 spectrum so teachers could learn how concepts were developed over time. 
However, the need to prepare elementary teachers in both subject areas led to the decision to 
have three separate courses: the existing one semester, three-credit Crusades course for 
mathematics teachers in grades 5-12; a comparable course in science; and an integrated, full- 
year, six-credit course for elementary teachers. 

Several of the interviewees noted that the shape of the state was instrumental in their project 
design: Arkansas is square, with the state capital, Little Rock, no more than 2 Vi hours driving 
distance from every part of the state. By dividing the state into a few “regions,” the AR SSI 
could offer professional development within easy driving distance of every teacher in the state. 

In developing the design to reach as many teachers as possible with high quality courses, the 
planning group stressed the importance of teaming people who had a deep understanding of the 
disciplinary content with people who had expertise in classroom teaching. Based on the 
experience in the pilot program, they were confident that the colleges and universities would be 
willing to provide graduate credit for the grade 5-12 science course and the integrated grade K-4 
course as well. Recalls a faculty member from one of the universities who has been involved 
throughout the process: 
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In 1991, when this committee was put together by Higher Ed, the whole deal was that 
whatever we do, she ( Diane Gilleland) had representatives from all over the university 
there. . . . The Arkansas Math Crusade, even before we got the SSI grant, was approved 
[for graduate credit] on all the graduate degree granting institutions in the state. And so 
we had that history, this collaboration that all the universities agreed to offer and give 
credit for this course. 

Providing Incentives for Teachers 

The AR SSI anticipated a number of likely barriers to the implementation of the program, and 
created strategies to help deal with them. One of the most important considerations was the need 
to provide incentives for teachers to participate in the Crusades. Previous efforts had found that 
teachers were reluctant to “give up their summer,” but were willing to attend professional 
development courses in the evenings during the academic year. Accordingly, the proposal 
planning team decided to offer the Crusades courses during the academic year, making sure it 
was engaging enough to keep teachers’ interest after a full day of teaching. 

One of the major incentives planned for teachers was to provide them with materials for their 
classrooms. The project worked out an arrangement where the schools had to provide matching 
funds. In addition, rather than paying stipends to teachers, the planning team decided to “entice 
them with graduate credits,” which would count toward the number of hours needed to move up 
a step on the salary scale in the districts. Again based on the pilot experience, the decision was 
made that teachers would have to take the courses for credit; teachers would be asked to provide 
$100 toward tuition, either from district funds or out of their own pocket, so they would be 
invested in the process. Said the Project Director: 

There was a lot of work to [the Crusades course.] And the first semester we tried it, way 
back in ‘92 they could choose to audit or take it for credit. [Those that decided to audit 
the course] refused to take a final or wouldn ’t write a paper. “I’m just auditing; 1 don 7 
have to do all of that. ” They would come late or leave early, very unprofessional. So 
after that semester we locked it down. You take it [for] graduate credit where we hold 
you accountable, and you must do all of these things or you don 7 get the opportunity [ to 
participate] . 

Building Infrastructure 

Arkansas already had a network of 15 regional centers, called Education Service Cooperatives, 
which were charged with providing an array of services to the schools in their area, but the 
planning team did not consider them an appropriate vehicle for delivering intensive, content- 
based professional development. Explained one member of the planning team, “Our 15 Coops 
were legislated to help our rural districts buy toilet paper, Coop transportation and Coop 
consulting services, but they were just generalists.” 

Accordingly, the decision was made that the mechanism for the professional development would 
be regional partnerships comprised of colleges, school districts, businesses, and other 
organizations. Each of the five partnerships to be established would incorporate all of the school 
districts from three of the 15 Education Service Cooperatives, and each would be advised by a 
regional council of key stakeholders in the area to help get local buy-in for the reform efforts. 
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The idea was that “through the regional partnerships and with help from the Academy for 
Leadership Training, the Project will train cadres of math and science leaders who will function 
at the regional level and with local schools,” working with teachers who complete the Crusades 
training programs to improve education throughout the state. A Steering Council of senior state- 
level policy makers would coordinate the efforts of the regional partnerships. 

In a further attempt to support teachers in improving their practice, the AR SSI would include a 
component aimed at administrators. The state had already begun to offer “Leadership 
Academies” for principals, and they wanted to “exploit it as best we could” by incorporating 
mathematics and science into the program. Said a member of the planning team: 

We realized that as teachers were completing (the Crusades courses), a lot of their 
success in the school and being able to use what they learned (would depend on their 
principals’ attitudes) ...and it made a lot of sense to expose school leaders at the 
Leadership Academy to the work (of the Crusades.) 

Fitting into the Reform Landscape 

The proposal mentioned a number of other existing initiatives that would also be incorporated 
into the AR SSI. Project STRIVE, an NSF-funded program that places mathematics and science 
teachers in real-world research settings in the summer would be expanded to a larger number of 
teachers, who would also be expected to design and implement “proactive knowledge transfer 
programs.” Federal funds allocated under the Carl Perkins Act would be used to expand the 
Applied Academics program into additional vocational and technical education programs in the 
state; and ARKnet and other technology initiatives would be used to facilitate communication 
among the regional partnerships and between each partnership and the schools it would serve. 

While the proposal language referred to reaching “all teachers” through the Crusades and the 
other project components, in interviews, planning team members noted that their more realistic 
hope was to engage a critical mass of people and create enough momentum so the reforms would 
radiate out from there. 



Implementing the AR SSI: 

Rolling Out the Crusades as Planned 

The AR SSI plan was to create a vision of quality mathematics and science education throughout 
the state and to use the Crusades courses as the primary vehicle to increase teacher capacity to 
implement that vision in their classrooms. To support that vision, the project planned also to 
work with administrators and others in the state to create an environment conducive to reform. 

Getting the Word Out 

Project leaders worked very hard right from the beginning to make the initiative as visible as 
possible. In fact, the Project Director notes that the first person she hired was a Communications 
Director, because once you get the grant, you have “to get the product out there in six months or 
less, [have] good publicity or people wonder what you’re doing.” 




The AR SSI Communications Director did a good job of coordinating efforts to get the word out. 
Said one of the project leaders: 

He was very, very active; he had a lot of media contact, and generated a lot of stuff. He 
went to see people. ...[We] had contact through the state public PR community with a big 
public relations firm that we could get information out there. ...I think one of the 
requirements for teachers with the Crusade classes was that they themselves write a 
letter to the editor of their local paper or sort of an equivalent of an OpEd piece to a 
small town paper about what they were doing. 

Communication was also facilitated by the fact that just about every one involved in Arkansas 
education seems to know everyone else: For example, “We were fraternity brothers in college. 
And we were elementary school principals together”; and “[The] math person in our department 
of education was a fellow college student with me during all of our college years.” In addition, 
communication between the ADE and the ADHE, historically strained, worked unusually well in 
the AR SSI. Recalls a long-time staff member in the ADE, “Those were some of the best times 
of collaboration between this agency and that agency.” 

Sticking with the Plan and Managing New Interactions 

As in any collaborative effort, the AR SSI encountered a number of difficulties in implementing 
its design, and had to decide when to stick to the initial plans and when to modify them in order 
to get around obstacles they encountered, to take advantage of emerging opportunities, or to 
accommodate others’ views. Although some components described in the proposal played a 
smaller role than anticipated as the initiative evolved (e.g., Project STRIVE to place teachers in 
research laboratories, and the Applied Academics Program), the AR SSI rolled out pretty much 
as anticipated. The persistence of project staff, and the momentum established over time, 
enabled the initiative to survive even major changes in leadership at the state level, including the 
1996 resignation of the Director of the ADHE, who had been a driving force behind the AR SSI. 

The Project Director spoke about how she learned to negotiate the terrain, including “borrowing” 
staff from districts as a win-win proposition, where the AR SSI benefited from the services of 
talented people and the district increased their capacity in the bargain. 

7 called the Little Rock school district and asked them if I paid her salary at Little Rock 
school district, will they let me borrow her for the life of the rest of the SSI. I’d pay them 
but they would have to give her to me. And then if there was a time or two that she’d 
need to go over and help them with something, fine. That was the best arrangement 
because the school district knew that they would get her back with additional skills and 
resources. And we were flexible about how we did the work. She loved it. She grew so 
much working on our national project to take back to Little Rock school district. I 
actually did that twice. [When a Partnership Director in the Delta left, I] asked the 
superintendent, “ Could I hire one of your central office people that was wonderful in 
math and science and has been with our Crusade for a year to come in to finish up 
Carolyn ’s partnership? I’ve got to make sure the SSI stays strong. I’ll pay her salary 
through your school district. You can train somebody else and get some other skills. 
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You ’ll have somebody coming back to you with additional skills. ” Beautiful 
arrangement; I learned to do these things for sun’ival purposes. 

Learning to negotiate the terrain was particularly important in the case of the Education Service 
Cooperatives. The fact that so many of the key stakeholders in the state were involved in the 
planning and implementation of the AR SSI created a widely shared sense of ownership. The 
one major exception was the Education Service Cooperatives, which were reported to be 
threatened by, and antagonistic toward, the efforts of the AR SSI. Said two key players in 
response to a question about whether there was any resistance to the efforts of the AR SSI: 

And there was some tension between the SSIs and [the Coops]. And part of it was pride 
of ownership within the regional service centers. Because they were doing a lot of one 
time in-service, short term and so forth. And anything that cut into their deed, they felt 
totcdly threatened by. 

They’re cdl ... run very independently, even of the state department of education. ...Once 
we got the grant, many of them were actively hostile because they thought we wanted to 
use “their” money for math. 



The AR SSI team “had to learn how to work the Coops” in order to enlist their cooperation, but it 
was an uphill battle. 

In the very beginning of that first year when we were having so much trouble with the 
Coops, I went before the directors meeting and said that one of our goeds is to find 
additional human resources that your Coops can have, so I’m proposing that we get 
technology specialists, early childhood specicdists and by the end of our SSI I’d like to 
have a math and science specicdist in every Coop. And they laughed and said, “You’ll 
never get that to give; we don 7 have the resources in the state. ”... They were always 
( negative ) whenever you came before them with any idea that was not their idea. 

A number of the AR SSI leaders were active in the state policy arena, and persisted in their 
efforts to take advantage of, and create, opportunities to further the goals of education reform, 
including strengthening the Education Service Cooperatives by adding “specialists” to their staff. 
By the end of the AR SSI, their efforts had helped in getting both technology and early childhood 
coordinator positions for the Cooperatives, and had set the stage for adding mathematics 
specialists in the future. 

Although there were many adjustments along the way in order to gain acceptance of the AR SSI 
and to accommodate local constraints, the initiative was unyielding in a number of areas. One 
place they stood firm was in insisting that teams comprised of a content expert from a college or 
university and a master teacher teach the Crusades courses. Project staff considered this team 
approach to be one of the strongest features of the AR SSI. Said one: 

For the first time we had higher ed at the play, there was a collaboration between public 
and higher ed, in terms of development as well as implementation. ... You couldn 7 say 
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higher ed was in charge, because there was a public school peer teacher there cdl the 
time. And it brought out a really neat mix when the teams functioned well together, of 
practical classroom experience with content base and some good strong pedagogy. 

Project staff also resisted requests from some faculty members to “send me the syllabus; I can 
teach any course if I know what the syllabus is.” When people expressed reluctance to go along, 
the AR SSI was able to use NSF for leverage. Noted one of the key players: 

NSF was cdways the silent partner, in that when you needed to do something or needed 
something to take place, you could cdways say, “NSF expects us to do this. ” Not that we 
shouldn 7 do it or didn 7’ want to do it necessarily, but it was always nice to say. If we ’re 
going to continue to take NSF money, then we need to do these things. 

Without exception, every one who wanted to teach a Crusades course was required to attend two 
weeks of training, and despite some initial grumbling, every one did. One of the key players 
noted that given the active involvement of the Director of Higher Education, you would expect 
cooperation from the public universities, but that private universities in the state were also on 
board, attesting to the broad support for the initiative. 

“Crusading” 

The Crusades courses were offered throughout the state, typically one night a week, with an 
emphasis on helping teachers “understand what an effective classroom might look like.” The 
grade K-4 Crusades course integrated mathematics, science, and (with support from state funds) 
language arts, in recognition of the fact that teachers responsible for self-contained classrooms 
would benefit from professional development across the curriculum. 

At the grade 5-12 level, articulation across the grades was a key focus, starting a concept at the 
fifth grade and showing how it was developed in the higher grades; observers noted that the 
inclusion of teachers from multiple grade levels led to conversations about how teachers could 
reinforce rather than duplicate the efforts at earlier grades. 

In interviews, project staff indicated that participation was greatest at the grade K-4 level, even 
though the integrated mathematics/science/language arts course was a full year. In contrast, the 
project began to “run out” of secondary mathematics and science teachers who were both 
amenable to reform and willing to devote the time required to participate in the single- semester 
Crusades courses. Explained a faculty member who taught both grades K-4 and 5-12 Crusades 
courses: 

When lam working with elementary teachers, they’re so hungry for “Can you give me 
one more way to explain this, one more visucd image, one more tactile thing?” Then as 
you move up to middle school, there’s some, “ Well I can see some advantage to doing 
some of this. ” And then you get up to high school, “Well that’s nice, but it would take up 
a whole day and I have cdl this curriculum to cover. ” 

The project also came to realize that providing materials to the teachers was a mixed blessing; 
while providing an incentive for teacher participation, and increasing the likelihood that teachers 
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would use course activities in their classrooms, purchasing large quantities of materials proved 
very expensive. Not only did the project spend a great deal of money on these materials, but also 
some schools and districts balked at the costs of providing matching funds. Recalled one of the 
key players: 

Everybody that came to the Math or Science Crusade went away with a big, big bunch of 
manipulative s. As it turns out the teachers were just delighted to get it, it didn ’t matter 
where it came from. And some administrators saw that as a very negative fix, because 
they didn ’t want have to put up for the material — put up the money. And that, over time 
may have precluded some of our smaller districts from participating. 

As the initiative unfolded, there was some feeling that the AR SSI might have “gotten ahead of 
the curve,” because the other parts of the system that were needed to enable teachers to apply 
what they were learning in the Crusades courses were not yet available. As a project leader 
explained: 

We had large numbers of teachers participate in courses, but the curricula that really 
were standards-based were not yet published. . . . When teachers went back into their 
classrooms, they were struggling with how to infuse these ideas within these old kinds of 
curriculum. So it was almost like we were moving in the right directions, establishing the 
need ,... [but] we were a step ahead of the support system. 

To facilitate the transition for teachers, the AR SSI arranged to pilot some of the standards-based 
materials under development nationally. They also worked to ensure that the rest of the context 
was as supportive of teachers’ efforts as possible. For example, over time, the AR SSI began to 
“focus on schools where the leaders had been through the leadership academy, or were willing to 
go through it,” both to pave the way for matching money and to provide the necessary 
administrative support for reform. 

There was some concern about whether this strategy of waiting until the districts were ready to 
participate would allow the initiative to serve the neediest districts, and in particular whether the 
AR SSI was doing enough to encourage minority participation. One of the key players recalls 
being approached by a regional partner who was a member of a minority group: 

[She said] “look out at the hundred and some people that came to this leadership 
conference, how many faces like mine do you see?” And I would say, ... “Tell us where 
to go to find them. Tell us where to go. ” 

A major part of the problem was that the pool of minority mathematics and science teachers was 
so small to begin with; the Project Director noted that while 25 percent of the Arkansas student 
population is black, only 16 percent of the teachers, and only 9 percent of the mathematics and 
science teachers are black. The AR SSI tried to recruit minority members who were in 
leadership positions at the district level, but without success, because the salaries in the state 
positions were not competitive. 
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While the AR SSI took the approach of “a rising tide lifts all boats,” there was some 
disagreement within the project leadership team about whether that was the appropriate 
approach. Recalls one of the key staff members: 

We had some heated discussions when we would get to equity issues. Because having 
come from Little Rock, which is an urban district and high minority population, my idea 
was that you can 7 deal with equity and wait for equity to come to you. Equity doesn 7 
mean that everybody gets equal treatment. Equity means that we reach out for potential 
and we nurture it. 

One of the ways the AR SSI addressed equity was through EQUALS workshops, including an 
ESL EQUALS program offered in 20 districts that serve large numbers of language minority 
students. The AR SSI also tried to reach out to the broader community to increase support for 
mathematics and science education reform through Family Math and Science programs, “trying 
to get communities and parents to understand what we were doing.” The project final report 
notes that more than 50,000 parents and students participated in these programs, including 
programs presented in Spanish in two regions that had large Spanish-speaking populations. The 
project also wrote and distributed a booklet entitled “It’s a Family Affair,” to help parents 
understand why and how mathematics and science instruction was changing, and providing ideas 
for activities to help their children “build strong math and science skills at home and have fun 
doing it.” 

The strategy of disseminating “booklets” on various topics evolved during the course of the AR 
SSI as a means of increasing the impact of the initiative, reaching people beyond those 
participating in the Crusades and Leadership activities, and facilitating ongoing work after the 
end of the NSF-funded period. Topics addressed in these booklets included equity benchmarks; 
guidelines for the selection of mathematics and science instructional materials; and the alignment 
of curriculum, instruction, and assessment. 



Making the Case for the AR SSI: 

“We just couldn’t get our story across to NSF” 

The AR SSI was highly visible in the state, with large numbers of teachers participating in the 
Crusades courses, creating a buzz about mathematics and science teaching. While there were 
some schools and districts less engaged than others, and reluctance on the part of some 
secondary teachers to engage in reform, there was a general perception within the state that the 
initiative was making a very important contribution to mathematics and science education. 

The AR SSI had a considerably more difficult time convincing people outside of the state that its 
efforts were on the right track. A major part of the problem was that the project found it 
difficult to communicate its strategy to NSF. The AR SSI saw itself as working hard to build an 
infrastructure for continued reform; NSF saw a large professional development initiative. 

Recalls the Project Director: 
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It was a lot of pressure. Every time I talked to [ the NSF Program Officer] he kept saying, 
“You ’re not a professional development project. You ’ve got to change policies. You 've 
got to work at the top level and change policies. ” Then I said, “Well, I know that, but in 
our state we have got to work at the grassroots level, and you ’ve got to work with the 
school board, and... you’ve got to have a product that proves itself. And it’s through 
professional development and our Crusade. ” At first he thought that we were writing 
curriculum for our students and I could never get across to him that we created modules 
for the teacher training. .. .They finally had to come down here and go visit some classes 
with me and look through the notebook and understand what we were doing. Anyway, it 
could have been my communication style. ... We just didn ’t get that story across to them. 

One of the problem areas was different interpretations of the “unit of change” to be targeted in 
the reform. Interviews conducted as part of this study suggested that the AR SSI had an implicit 
strategy of getting enough teachers on board that superintendents who contemplated reform 
would find support rather than resistance at the ground level. However, they were not able to 
explicate this strategy to NSF. 

We would always say that the focus of our training is our teachers. Until you can talk to 
teachers and get teachers doing what they ’re supposed to in the classroom, you ’re not 
going to make changes in the school or district, either one. And of course that sent them 
up the wall. 

The project’s response to concerns about the teacher as the unit of change was to declare the 
district the unit of reform. Acknowledged one of the key players, “We would change our 
verbiage from time to time, but we really didn’t change what we did.” 

Issues of equity were particularly contentious. Although the proposal and the documents 
produced in the project included equity as a major focus, it appeared to NSF and to external 
reviewers that equity was not as central as it needed to be, perhaps because there was no extra 
effort specifically for the high minority school districts. These criticisms seemed to baffle the 
SSI leaders, who emphasized that the equity issue in Arkansas was more along SES than racial 
lines. 



[Equity] is an area where I think we probably had an ongoing dispute with [NSF] that 
got pretty bad at times. We saw equity not as they did, as a minority issue in a state 
where the only issue was minority. For us also equity was poor kids, white or black. 

They had two or three key questions that they kept pounding in on. And we didn 7 
understand where the questions were coming from. One had to do with minorities and 
minority participation in everything that we did. . . . That’s not our problem in Arkansas. 
It’s not minorities; it’s socio-economics. It’s poor children. That’s our problem. So we 
have to look at ways to get families to understand about math and science. And to get 
our young girls that are 14 and 15 that have babies and are single mothers. So that was 
one thing that kind of irritated NSF I think. 
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Another area of contention was the project terminology. “Crusades” as the title of the 
professional development initiative worked just fine within the state, but it didn’t play well to a 
national audience. Recalls one of the key players, “Every time we went to NSF somebody would 
criticize our word Crusade. ‘Why do you use the word Crusade?’ It works for us. They didn’t 
like that word. We got that every year.” 

In contrast, the project was in full agreement with another concern expressed by NSF: the lack 
of data to demonstrate that the AR SSI was having an impact on teaching and learning. 

I think in the beginning we didn ’t think too much about assessment at all and we didn ’t 
plan too much about how to gather data in the beginning. We were all neophytes. ... We 
had a lot of trouble the first two or three years trying to figure out where was the data, 
how do you get it, what does it mean and still what we would present to NSF wasn 7 
enough. So that was the piece that by the end of the SSI we were realizing some things 
that we should have done early on. 

Project staff tried to get data from the state department of education, but hit a brick wall, with 
Department staff claiming that release of data would violate student privacy laws. Even getting 
the Governor to intervene didn’t succeed in getting them the information they needed to assess 
the impact of the AR SSI, although these conflicts “led to the Director [of ADE] being fired 
eventually.” Said the AR SSI Mathematics Director: 

We couldn 7 track other than what districts were volunteering to track for us, the kind of 
convincing data, hard data pieces that we need. We tried, but we can 7 tell a district to 
give us student data. They can volunteer to do it, but ...we never got a handle on it. 

The project team was clearly frustrated by their inability to communicate with NSF. Recalls the 
Project Director: 

We couldn 7 figure it out. We are all fairly intelligent people and we sit together as a 
team, “How should we do this next presentation ? How should the next site visit go ? What 
do we need to do to tell our story?” We all worked together on it. We even practiced our 
presentations in front of each other. 

In retrospect, project staff wondered if they erred in not “bragging” more about their 
accomplishments, noting that people from other SSIs were more open in taking credit for 
activities in their states. 

We couldn 7 cdways take direct credit for it because we’re kind of infused in the process. 
And if you stand up and take direct credit for something then the Coop, state department 
and school district themselves ( might be offended.). It’s a team thing and we cdways 
talked about it as “we ” and “our. ”... In Arkansas we ’re not prone to brag about 
anything that we do. Southern hospitality is such that you don 7 talk about yourself. You 
don 7 brag about the good things that you do. That’s how we ’ve cdl been brought up. 



15 




Interestingly, the Project Director noted that Diane Gilleland, the Director of the ADHE, coached 
her and other project staff about the difference between communicating within and outside the 
state: while it is important to give credit to each of the players in the state, “when you go to NSF 
you tell it as the SSI story.” This was difficult advice for the AR SSI to implement; said the AR 
SSI Project Director, “we probably didn’t brag enough.” Accordingly, key players in the AR SSI 
were “disappointed, but not surprised” that the AR SSI did not receive funding for Phase II, 
noting that NSF never “truly understood what we were doing in Arkansas.” 

In contrast to their communications during the funded period, the final project report to NSF is 
quite direct in highlighting the successes of the AR SSI, noting that the statewide systemic 
initiative had “set in motion major changes in the state’s educational system that are instrumental 
in significantly improving student learning in mathematics, science and technology. ...Arkansas 
now has a system that emphasizes interconnectedness, active learning, shared decision-making, 
and higher levels of achievement for all students.” The accomplishments of the AR SSI during 
the funded period, and the ongoing impact of that work, are discussed in the following section. 



Accomplishments of the AR SSI 

The AR SSI worked to change the system at a number of levels, including (a) increasing the 
capacity at the local level by providing professional development to large numbers of teachers 
and working with school and district leadership teams; (b) establishing regional partnerships to 
provide ongoing assistance to local districts; and (c) creating a more supportive policy context 
for improved mathematics and science education. 

Increasing Local Capacity 

The project’s final report indicated that more than 8,000 teachers participated in one of the three 
Crusades programs, and that these teachers “impacted the learning of 87 percent of Arkansas’ 
students.” In addition, the proposal for Phase II funding indicates that many district, school, and 
community leaders participated in the initiative’s leadership development programs “to build 
capacity for school-based reform”: 

More than 4,000 school leaders — teachers, administrators, school board members, 
parents, other community leaders — have completed the state Leadership Academy of 
specialized leadership institutes for teachers, principals, or teams. Approximately 70 
percent of the superintendents have completed at least one Leadership Academy. 

The AR SSI leaders interviewed for this report used varying lenses in assessing the impact of the 
initiative on teachers and students. One important result they noted was increased expectations 
that all students can leam challenging mathematics and science when given the opportunity to do 
so. Some of the most positive comments had to do with the magnitude of the effort and the 
resulting enhanced understanding of what constitutes quality mathematics and science 
instruction. 

Nobody had ever done a statewide math and science effort in our state, ever. To have 
gotten thousands of teachers to devote all of those hours to professional development, 
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and we were able to get a lot of money out of the Department of Ed to buy the materials 
and manipulatives. Nobody had ever done that before. 

I think it was overwhelmingly successful. ... We created almost a need for professional 
development in math and science. And we trained a large number of teachers and gave 
them some introductory content in terms of the stanclards-based program and instruction. 
I think it heightened the awareness on the part of elementary teachers. We ’re doing a 
better job teaching mathematics beyond rote content. 

Others described the impact of the AR SSI primarily in terms of changing the discourse about 
mathematics and science education. Said one: 

We established a common vocabulary; we established an understanding of linkage with 
the state frameworks and how it translates into assessment and performance tasks. 
...Teachers can sit down at a table, whether they agree or disagree about the changes, 
they have a way to have conversations. ... When you go through some of these hills, and 
you hit this rural school, and you walk in and you say, “Can anyone talk to me about 
what you mean by standards in mathematics?” and they have a working knowledge of it. 

The consensus appeared to be that the AR SSI did a lot to create awareness and foster dialogue, 
but that five or six years was not sufficient to change deeply ingrained behaviors. Said one of 
the key players: 

It was a good start. But. . .1 don ’t think we have the level of classroom practice that we 
had hoped we would have. ...Well, these people have learned bad habits, some of them 
over 20 years, the way they were taught. And they’re not going to have one course and 
be able to put all of that stuff together. 

Establishing Regional Partnerships 

In addition to serving teachers and schools, the AR SSI had as one of the goals of the regional 
partnerships that there would be changes in the university culture, with more attention paid to 
quality teaching. While this kind of impact was by no means universal, there were encouraging 
signs of change in the universities. Said one observer from the Arkansas Department of Higher 
Education: 

By having all of these courses team-taught, we began to graducdlv change the university 
culture. We were bringing talented K-12 teachers into the university really to role model 
to faculties who were willing to learn from them. So that was I think a huge plus. 

A faculty member reported changes she has seen in at her institution and others, including 
increased use of “basic calculator technology” in college algebra courses and more attention to 
connecting the mathematics taught at the college level to the mathematics taught at the grade K- 
12 level. 

There are lots of universities here in Arkansas that are still using Math Crusades 
materials as part of their undergraduate methods courses. Maybe they haven 7 lifted the 
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whole course verbatim, but it’s provided some impetus for them to change the way they 
do teacher training. 

Although given the number of faculty involved in teaching the Crusades courses, there was some 
disappointment about “the lack of fundamental change in teacher preparation programs,” the 
prevailing view appeared to be that the AR SSI had made considerable progress in developing 
important connections so higher education could continue to serve as resources for the schools, 
and in creating a number of “pockets of change” at the university level as well. There was also 
general agreement that one of the major impacts of the AR SSI was to forge relationships, not 
only between the colleges and schools, but also between the mathematics and science teacher 
organizations, and between the ADE and the ADHE. Joint meetings of the teacher associations 
initiated by the AR SSI have continued, and there is an annual state equity conference co- 
sponsored by the ADHE and the ADE. 

Creating a More Supportive Policy Context 

Project leaders recognized that state policies would have a major impact on their ability to 
sustain and expand the changes initiated in the AR SSI, and they devoted considerable effort to 
aligning state policy with the reform goals. The project’s final report lists a number of changes 
in state policy passed by the Arkansas General Assembly and/or by the State Board of Education, 
including increased graduation requirements in mathematics and science; revisions in the 
textbook purchasing regulations to allow the use of state funds for mathematics manipulatives 
and kit-based science programs; and required professional development for teachers to renew 
their licenses. The SRI case study released in 1998 concluded that “taken as a whole, the AR 
SSI’s impact at the state level was impressive.” In interviews, project staff reported more recent 
policy changes that were set in motion by the SSI, including creation of a mathematics/science 
middle school license, and the development of “bench marking tests” in mathematics for grades 
4, 6 and 8, Algebra I and Geometry. 

Explained the Project Director: 

The policy changes came slowly and pretty much were building in that last year we had 
the SSL And now we have our Smart Start program, our math specialists, this fourth 
year of math, continued scholarship money [for prospective teachers], especially when 
you ’re in the math and science field. We have a built an infrastructure that was because 
we had that SSI. 

The AR SSI also helped to shape the state’s professional development “system.” Not only are 
Arkansas teachers now required to earn renewal credits for recertification, but also the 
understanding of what constitutes quality professional development has changed. Said these two 
observers: 

I think what the SSI did was it showed administrators that you didn ’t fix the problem with 
a one day math workshop. ...I think we educated them here in the SSI that’s what the 
research said about changing student improvement, that it’s a multi year process, that it 
has various components. I think that’s the thing that the SSI did that no report could ever 
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reflect is that it educated administrators about what meaningful staff development would 
be. 



Had SSI not come along, we would never had had college and university folks and public 
school folks coming together to talk about what professional development needs to look 
like.... We very seldom do training now that’s one time; it’s sustained long-term. 

Sustaining Reform 

Although some of the activities conducted during the AR SSI were scaled back when NSF 
funding was no longer available, some components of the initiative have actually been expanded 
since the SSI. First, the number of regional centers was increased from 5 to 10. Second, the 
state decided to fund 26 full-time mathematics specialists, one at each of the 15 education 
Service Cooperatives, targeted primarily at the elementary level, and one at each of the 10 
regional centers. The Project Director describes how the mathematics specialists came to be: 

At the end of the sixth year I went to our [ADE ] director. . . . And I said, “... I have an 
idea about how we could keep the SSI going. I want to write it up into our plan. . . . We 
need 26 math specialists in the state. You could do just 15 now [for the Cooperatives] 
and add more of them later, ...11 more for the university. One would be elementary and 
10 secondary. ” We ’d do it in phases and 1 laid out what the scenarios were. “ But the 
most expensive one, and the best one we ’ll put them all in at the same time. We ’ll train 
them cdl and get going here in our state. ” He loved it. 

It is particularly interesting to note that while the AR SSI is no longer an official entity, the 
Mathematics Director coordinates the training and deployment of the 26 mathematics specialists, 
describing what has happened to continue the activities begun in the AR SSI as “what systemic 
change is all about.” Similarly, the Project Director continues to coordinate the 10 Centers: 

I ccdl myself the coordinator. I’m unofficially the boss. 1 try not to say that. Once every 
six weeks or so I call a directors’ meeting, the directors of cdl the centers. And we come 
together and share what we ’ve been doing in our different regions. We have two or three 
state projects a year that we work on together. They all have assignments. I go visit 
them. I guess it’s psychological; I still have the role of the director of SSI. People still 
say that. 1 just carry that on. That’s kind of a perception that I just haven 7 cdlowed to let 
go on their part. 

Although the state did not fund science specialists, as it turned out, the majority of the Center 
Directors have backgrounds in science, and both science and mathematics activities are being 
provided. Some of the Centers are continuing to offer the Crusades courses, per se; others have 
begun to “specialize” in a particular area, such as helping districts align their curricula with state 
standards and select instructional materials that best fit their contexts. According to one of the 
Center Directors, the districts “got used to us coming in and doing stuff during the SSI days,” 
and have continued to ask for help since that time. 
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Factors that Shaped the Arkansas SSI Story 



The AR SSI was a key player in mathematics and science education in the state, fostering 
widespread awareness of the need for reform and providing professional development on an 
unprecedented scale. A number of factors contributed to their successes, including putting in 
place policies and service delivery mechanisms that are continuing to impact mathematics and 
science education statewide, and to their frustration in not getting support from NSF to continue 
their work for a second phase. 

> The SSI came at the right time for Arkansas. 

Education improvement was at center stage in the state in the early 1990’s, with a major 
legislative initiative to improve education generally, and mathematics and science education in 
particular. The AR SSI had active support from the Governor and from education leaders in both 
the Arkansas Department of Education and the Arkansas Department of Higher Education; there 
was consensus that major improvements were needed, and that this initiative would make a key 
contribution to those improvements. 

> AR SSI leaders understood the importance of being highly visible. 

The leaders of the AR SSI recognized that if people don’t know what you are accomplishing, 
they would quickly begin to question the value of an initiative. Hiring an experienced 
Communications Director at the outset was important, as was the fact that there was a single 
“signature” initiative, the Crusades, to help provide name recognition. It would have been 
difficult to find a teacher or administrator in the state that had not heard of the Math and Science 
Crusades. 

> The interventions were well received. 

The Crusades courses were generally considered to be well designed and well implemented. 
Project leaders were firm in their insistence that the courses be taught by teams of content and 
pedagogy experts, and that instructors attend two-week training programs to help ensure that the 
courses would be implemented as planned. Providing participants with classroom sets of hands- 
on materials was a major plus from the teachers’ perspective, and the Principal Institutes and the 
Leadership Academies for district teams helped ensure administrative support for their efforts. 

> The AR SSI was never able to get its story across to NSF. 

Although highly successful in its communication efforts within the state, the project was unable 
to tell its story to NSF, or at any rate to convince NSF that the AR SSI strategy was a reasonable 
one. Project leaders spent an inordinate amount of time trying to figure out how they could show 
NSF how much they were accomplishing, but were never able to do so. Disagreements about the 
unit of change, and the extent of focus on equity, might have been resolved if the initiative had 
data to show an impact on student performance, and especially on “narrowing the gap,” but the 
AR SSI did not have systems in place to produce such data. 

> The AR SSI continues to have an influence in the state. 

As key leaders in mathematics and science education in the state, AR SSI project staff were 
integral in a number of the policy changes initiated during the funded period, including increased 
graduation requirements, instituting requirements for teacher recertification, and allowing 
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districts to use state textbook funds for purchasing manipulatives and kit-based programs. 
Moreover, as a result of their efforts the state has devoted a great deal of additional resources to 
mathematics and science education improvement; there are now ten regional centers devoted to 
mathematics and science education, and 26 full-time mathematics specialists statewide, all 
supported with state funds. Although the AR SSI did not receive NSF funding for a second 
phase, the Project Director continues to coordinate these efforts, and statewide systemic reform 
in mathematics and science education continues. 
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Appendix H 



Case Report 

Maine Statewide Systemic Initiative 




Introduction 



The National Science Foundation (NSF) funded the Maine Statewide Systemic Initiative (ME 
SSI) from 1992 tol997 for a total of $10 million. Maine was among the second cohort of states 
funded under the Statewide Systemic Initiative Program. The fiscal agent for the ME SSI was a 
newly created non-profit organization called the Maine Mathematics and Science Alliance which 
has been sustained as a focal point for mathematics and science education reform in the state 
beyond the funding of the SSI. 

Talk to anyone in Maine about the education system in the state and you can count on hearing 
the words “local control” almost immediately. As two SSI leaders described the nature of the 
state education system in research interviews: 

Because it was a local control state, we coulcln ’t say, “You should use these materials or 
you should use this approach, or you should do anything. ” In particular we could only 
talk in relative generalities about good curriculum development. You should have a K- 
12 committee, you should have school board, you should have parents, it should be 
assessing what happens and finding out what isn ’t happening. (SSI Co-PI) 

In the context of designing, implementing, and evaluating statewide systemic reform in Maine, 
“The tradition of local control ... particularly strong in Maine’s public educational system [was] 
to be recognized as both a liability and asset in the initiative.” {Maine, A Community of 
Discovery , 1991) In order to understand systemic reform and the SSI in Maine, it is essential to 
understand this commitment to local control and the state’s unique demographics. 



Ten Years Ago: 

The Context for Mathematics and Science Education Reform in Maine 

The bulk of the population of 1,200,000 in Maine is concentrated in the southeastern corner of 
the state, or “Down East,” as the locals call it, which is roughly defined as the region south of 
U.S. Interstate 95. In this area of the state are located the largest cities in Maine — Portland and 
Augusta. Although Maine’s largest metropolitan area around Portland numbers only 265,000 
people, it is very large in comparison to the myriad towns throughout the state with only a few 
thousand or a few hundred residents. Many of those communities are found on the winding, 
rocky coastline and multitude of islands famous for lobster fishing, lighthouses, vacation 
getaways, and spectacular scenery. Others are located in the geographically large, but very 
sparsely populated interior of the state, which depends largely on an agricultural economy. 

In 1990, among the diverse cities, towns, and communities in Maine were 183 school districts, 
employing about 14,500 teachers, and serving about 218,000 students. Communities served by 
Maine’s school districts ranged in size from as many as 7,680 students served in Portland to 
fewer than 20 students served in towns such as Bancroft or Beddington. Like other New 
England states, the system of education in Maine reflected its origins in colonial days. The 
devotion to local control in the education system in the state meant that each of those local 
education agencies had operated almost autonomously in matters of policy, management, and 
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instruction for more than two centuries. A few measures of centralized guidance, not control, 
had come about in the state in the 1980s, but each local agency in Maine continued to make the 
majority of its own decisions about policy and practice. Racially, Maine remained among the 
most homogenous of states, with a population nearly 98 percent white. Its largest minority group 
was Native Americans, with small populations of African-Americans, and Asians and Asian- 
Americans in its cities. 

The Origins of the Maine Mathematics and Science Alliance 

A small group of K-12 educators, higher education faculty, business and industry, and research 
representatives had been meeting in Maine for a few years with support from the Mathematical 
Sciences Education Board when NSF issued the Statewide Systemic Initiative Program 
solicitation. Another larger group called Partners for the Advancement of Science and 
Technology Education, involving businesses and educators, had also been meeting to identify 
connections and gaps between business workforce needs and K-12 education in Maine. With 
strong support from the Commissioner of Education and efforts of Department of Education 
leaders, members of these groups were brought together at a meeting in Augusta in 1991 to 
consider a response to the SSI solicitation. One of the first decisions the group made, in 
recognition of the diversity of local and broad interests within the state was to expand the 
planning group for the SSI to include additional representatives and stakeholders of K-12 
education, including local education agencies; business, industry, and research institutions; and 
higher education, both public and private. 

In 1991-92, when the SSI plan for Maine was being crafted several education improvement 
programs were underway in Maine that were viewed as directly or indirectly supporting the SSI 
plan. “An Act Relating to Restructuring the Public Schools” was passed in 1991 to encourage 
and support changes in school structuring and policies to improve teaching and learning 
opportunities in Maine schools. The act mandated annual redistribution of $700,000 in 
Department of Education funds and an additional allocation of $300,000 to support restructuring. 
The Commissioner implemented restructuring efforts to improve student outcomes and 
performance in support of the Act. Other policies in place in 1991 included Maine’s Common 
Core of Learning, an early curriculum standards report. The Common Core of Learning 
provided a common vision for reform in the state, which was “guided by the national standards.” 
Already in its sixth year of implementation was the Maine Educational Assessment (MEA), a 
low-stakes, statewide testing program that included science and mathematics and reported annual 
learning outcomes in grades 4, 8, and 11. In 1991, MEA used multiple-choice and open ended 
items and was being expanded to include performance and attitudinal items. Collection of 
demographic information for purposes of disaggregation of results was also to be included. 

The Department of Education had begun supporting statewide meetings three times a year for 
mathematics and science teachers and educators under the name Problem Solving in Science and 
Mathematics (PRISM). An SSI Co-PI described the conferences: 

We had teachers who taught teachers in math, so we helped empower budding leaders to 
be leaders and to have respect, and we always had a full house. We took them to a nice 
place for three days or two and a halfdays, and they investigated different ways to do 
math and science with their colleagues and in most cases listening to their own 
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colleagues present to them. . . . We also looked at professional development, . . . what were 
the key things in professional development we need to do as leaders of this effort ? 



Additionally, ten schools in the state had joined national efforts to improve educational policies 
and procedures as part of Maine’s Re: Learning program, dovetailing their efforts with the 
national Coalition of Essential Schools and the Education Commission of the States. 

Independent groups were also supporting educational reform in Maine, including the Maine 
Coalition for Excellence in Education, a partnership of business, education, and community 
leaders. Governor John McKernan instituted the Commission on Scientific Literacy to develop 
an action plan by the Spring of 1992 to increase science literacy of Maine residents, led by a 
broad-based citizen group. Finally, the SSI viewed the NSF-sponsored Maine Experimental 
Program to Stimulate Competitive Research (EPSCoR) as a valuable resource for improving 
scientific literacy. 

The planners of the SSI identified several needs the SSI would address in Maine. These included 
needs in K-12 schools, such as more innovative/non-traditional science and mathematics 
curriculum and instruction; improvements in students’ conceptual understanding of science and 
mathematics; increased student enrollment in science and mathematics at higher levels of 
education; greater access for K-12 educators to science and mathematics experts; and more 
qualified mathematics and science teachers. The identified needs also included changes in 
policies, such as better alignment of teacher certification requirements with standards set by 
national organizations; an improved assessment system in mathematics and science that 
measures conceptual understanding; and statewide curriculum frameworks. Additionally, the 
SSI planners identified needs in the state education system’s support structures, including more 
and improved staff development opportunities for mathematics and science teachers that 
emphasize problem solving or interdisciplinary instruction; increased capacity for interpreting 
assessment findings; a process to evaluate progress within the education system in terms of 
teacher certification, public support, implementation of standards, and resource allocation. 
Finally, increased public support for mathematics and science education was needed. 
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The Maine SSI Plan: 
A Beacon for Change 



The National Science Foundation funded the Maine Statewide Systemic Initiative for five years 
beginning in 1992. Priority needs the plan addressed included: 

• Improving professional opportunities for teachers through more time and resources 
for their own education in science and mathematics, increased involvement in 
governing teachers’ organizations, and greater access to science and mathematics 
experts. 

• Increasing the capacity/awareness of policy makers (administrators, local, and state 
leaders) so they will support educational improvements. 

• Improving the curriculum in science and mathematics so that it develops students’ 
conceptual understanding and problem-solving skills. 

• Providing inquiry-based learning environments in the form of schools, laboratories, 
and outdoor settings. 

• Developing community support (parents, employers, mathematicians, scientists, and 
government officials) of education reform. 

• Involving business and industry in education reform through fiscal and programmatic 
opportunities with the goal of developing a more competitive work force. 

• Improving educational opportunities in mathematics and science, and increasing 
enthusiasm and achievement in mathematics and science education for all students, 
and particularly for female students; non-college bound students; students from rural, 
isolated, and island communities; students from lower socioeconomic groups; and 
students of Native American, African-American, and Asian descent. 

The ME SSI plan comprised four major components organized under the Maine Mathematics 
and Science Alliance (MMSA), a new non-profit organization which became synonymous with 
the ME SSI during the life of the program. The components were: (1) curriculum and 
instruction, (2) community integration, (3) professional preparation, and (4) systemic planning 
and evaluation. 



The Technical Strategy of the Maine SSI 
The Maine Mathematics and Science Alliance 

In recognition of Maine’s history of local control and polarization, instead of appointing an 
existing agency, the SSI created a new entity to guide the initiative in the form of a non-profit 
organization. The rationale behind this decision was that “historical sensitivities and ownership 
of programs and policies” would taint the selection of any one agency to lead the SSI. The role 
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of this new non-profit organization was to serve as an umbrella organization and “catalyst for 
change.” The SSI Project Director described the importance of this decision in retrospect: 

The importance of the SSI in Maine choosing to go to a non-profit structure was very 
valuable, because it allowed that group to be flexible, to shift and change, and not 
necessarily be politically aligned anywhere. It is a high risk strategy in the sense that we 
could be disregarded, but you aren 7 pigeonholed anywhere. So you can take the high 
road without having to get dragged down by other values, you know, in the particular 
institution. ... I think the insight of starting this organization was very good. 

The Pis and Co-PIs initially provided leadership for the Maine Mathematics and Science 
Alliance; a broad-based partnership of education, business, higher education, research, and 
community leaders comprised the membership. The roles of the alliance were to provide a 
central office of operations, to serve as the policy-making body for the SSI, and to administer 
financial resources of the initiative. Its responsibilities included convening the committees 
leading each of the major components of the initiative and overseeing the initiative’s work in 
curriculum standards, instructional improvement, professional preparation and development, 
equity issues, community education and involvement, program evaluation, and ongoing strategic 
planning and assessment. An Executive Board of Directors was appointed to oversee MMSA 
operations. The Executive Board sought input from a National Advisory Council comprised of 
educators, policy-makers, mathematicians, and scientists outside of the state. Standing 
Committees were created to lead four major components of the initiative (Systemic Planning and 
Evaluation, Curriculum and Instruction, Community Integration, and Professional Preparation). 
The Standing Committees employed the expertise of business, education, and research leaders in 
the state. 

Curriculum and Instruction 

MMSA’s Curriculum and Instruction Standing Committee was responsible for developing and 
evaluating new mathematics and science curriculum standards and frameworks for grades pre-K 
through 16 that were aligned with national standards and Maine’s Common Core of Learning . 
The committee’s aim was to develop a flexible curriculum framework in accordance with 
Maine’s Common Core of Learning, NCTM’s Standards, Project 206 l’s Benchmarks, and other 
national curriculum documents that would aid in the alignment of curriculum across the state. 

A second goal for the Curriculum and Instruction Standing Committee was to align instructional 
methods with the revised curriculum. Development, testing, and dissemination of new materials 
and methods were to occur in model schools known as Beacon Schools. Beacon Schools were 
intended to serve as “model opportunities for in-house and visiting teams of teachers, student 
teachers, mathematics and science facilitators, school administrators, policy-makers, parents, and 
community members to develop, practice, observe, and evaluate new strategies for teaching 
mathematics and science.” One Beacon School, likely chosen from among those already 
involved in national reform efforts — Re: Learning, Coalition of Essential Schools — was to serve 
as a pilot in the first year of the initiative to develop and test the process of creating Beacon 
Schools. Six additional schools were to be added in the second year of the initiative. 
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The SSI used an application process with detailed review criteria for selecting Beacon Schools. 
Building infrastructure for reform within the school was inherent in the selection criteria, as was 
the development of infrastructure for scaling-up reform statewide. The former consideration was 
addressed by inviting applications from schools at varying stages of reform, so long as the 
schools conducted self-assessments to understand their current stage of reform, “demonstrated 
commitment to restructuring from a significant number of teachers and key administrators,” and 
had “a history of, or demonstrated potential for, parent and community involvement and interest 
in school issues.” A Beacon School application would include a plan for broader and deeper 
reform in mathematics and science for all students, involving a broad array of community and 
professional partners. The latter consideration was addressed by specifying selection of Beacon 
Schools representing different regions of the state, with the collection of regions covering the 
state. Moreover, Beacon School applicants had to commit “to participate in the systemic 
components of the initiative, sharing its discoveries and accomplishments with other schools 
[and] supporting professional development programs.” Equal consideration would be given to 
districts that proposed a single Beacon School and those that proposed a cooperative system of 
multiple schools within the district. 

Each Beacon School site would employ one Mathematics Facilitator and one Science Facilitator. 
The Facilitators were to be hired by the Beacon School, but paid by MMSA. Their main 
responsibility was to assist teachers with implementing innovative instruction in the Beacon 
Schools. They were to gather resources for teachers in the Beacon Schools and to aid in the 
design, dissemination, presentation, and revision of curriculum, laboratory, and field materials. 
They would also “pay particular attention to... instructional strategies for students who are not 
interested in technical or scientific careers,” which was noted as a target for equity in Maine. 

Facilitators and teachers at the Beacon Schools were expected to participate in professional 
development offerings of the SSI. After some time, the facilitators and identified lead teachers 
from each Beacon School would also serve as resources for other schools within each Beacon 
School site’s region to provide leadership and support for implementing new curriculum and 
instructional strategies in a growing number of non-Beacon Schools. Facilitators were identified 
from among participants in statewide leadership development meetings, members of the state’s 
mathematics and science teacher professional organizations, and an open application process. 

The thinking behind the design of Beacon Schools with Facilitators was described by one of the 
developers of the original proposal: 

Working very hard in certain key schools, ... I don ’t reccdl anyone not feeling that was a 
good way to go. I certainly felt very strongly that was a better way instead of trying to 
just do some broadly dispersed, let’s try to help math and science in the state of Maine by 
trying to do programs that reach everyone. I really felt that the way you have to do 
change is you work reed hard in a few places where you can really get some change and 
then . . . those give some models and something that other people can look to and then 
further expand what’s been done. I mean the whole idea of the Beacon Schools would be 
... standards-based ... really bringing certain schools, those who were willing to apply to 
be part of this program, to really get the standards working there and get some real 
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positive change, a change in the culture, which is so hard to do, recdly put a lot of effort 
in it. 

Professional Preparation and Development 

The Professional Preparation and Development Standing Committee was responsible for 
activities targeting preparation of pre-service teachers and professional development for in- 
service teachers. The committee oversaw four principal activities — the Beacon College, 

Institutes and Workshops, Teacher Support Staff, and Communications. The SSI planed to 
designate a Beacon College, analogous to the Beacon Schools, where pre-service programs 
would be developed and tested. The goal of the Beacon College would be to improve 
certification requirements and the recruitment of new teachers as a model for all teacher 
preparation programs in the state. In addition, student teachers from the Beacon College were to 
be placed in Beacon Schools during their pre-service training to gain exposure to new curriculum 
and instructional methods. 

Mathematics and science teachers at all levels throughout Maine would be invited to apply for 
participation in SSI-sponsored institutes to be led by Beacon School facilitators and other 
professionals from the research, higher education, and education communities. The institutes 
would be conducted for four weeks in the summer, with academic year follow-up activities, 
including site visits to Beacon Schools. Participating teachers were expected to change their 
own classroom practice and to share their experiences within their schools and districts. The SSI 
explicitly planned to use the first academic year of the granting period to plan the summer 
academies and follow-up activities, suggesting an understanding of the need for extensive 
planning of intensive professional development experiences for teachers. 

The teacher support staff and communication activities of the Professional Preparation and 
Development Standing Committee were related to increasing opportunities for teachers to 
develop curriculum and instructional expertise. First, the standing committee was to hire 
additional teaching staff to serve as permanent substitutes to teachers participating in MMSA 
professional development opportunities. This activity was meant to facilitate teachers’ ability to 
take leave from the classroom to expand their own teaching capacity. A long-range goal of this 
component was to develop teacher sabbatical programs. Experienced mathematics and science 
teachers, as well as professionals from the community, might be hired as permanent substitute 
teachers. Second, to facilitate communication among teachers, an interactive television network 
supported by the University of Maine would be used as a networking tool. The necessary 
hardware, software, and training for this system were to be provided to all Beacon Schools. 

Community Integration 

The Community Integration Standing Committee was responsible for overseeing programs 
aimed at increasing community awareness and support of mathematics and science education 
reform. The SSI was particularly interested in increasing support for programs aimed at 
traditionally underserved students. Local Community Action Teams, based at individual schools 
or in districts, would be comprised of parents, business leaders, educators, students, and 
professional mathematicians and scientists. This model was chosen largely because of Maine’s 
climate of strong local control; the Community Action Teams were intended to align their efforts 
with the priorities of local school boards and school administrative districts. 



7 




Community Action Teams were modeled after two programs already underway in the state, 
Leading Mathematics into the 21 st Century and the Maine Aspirations Foundation. The SSI 
planned to involve individuals from these existing programs in the development of Community 
Action Teams. Beacon Schools would be the first to develop Community Action Teams as a 
model before expanding the activity to other communities throughout the state. The teams were 
to be responsible for implementing public education programs and student apprenticeships, and 
for identifying a “broad-base of financial and human support for all components of the 
initiative.” 

Community Action Teams would be charged with identifying the needs of underserved groups of 
students in their communities and developing programs to address those needs. The SSI left 
decisions about which programs were most appropriate for each community up to local leaders. 
However, apprenticeship programs with businesses and other local partners were to be a feature 
of the services to underserved students in all communities. In the apprenticeships students were 
to be provided with first-hand experiences applying science and mathematics to real world 
problems. This program was to dovetail with the NSF-funded Maine EPSCoR Program, and 
become available to secondary and post-secondary students. 

Public awareness campaigns were to be the primary method used to gain community support for 
reform. These efforts included: holding public forums, implementing media campaigns, 
distributing written materials, lobbying local decision-makers, and networking with community 
leaders. The specific goals of the teams were to: increase local budget allocations, obtain in- 
kind and cash support for specific programs, and seek charitable contributions to MMSA. 

In addition, a statewide public education forum was planned for each year involving members of 
the National Advisory Committee and in year four a national conference would be held. 
Meanwhile, SSI program components would be showcased at other education conferences at 
regional, state, and national levels. 

Systemic Planning and Evaluation 

The Systemic Planning and Evaluation, chaired by a highly involved business leader, was an 
integral component of the SSI plan. Initially, the committee was to collect information on 
evaluation programs to aid in the design of the of SSI evaluation. In addition, they would collect 
and analyze data on several aspects of the education system to inform planning and 
implementation, such as: policies, legislation, teacher qualifications, resource allocation, equity 
issues, curriculum, citizen values and attitudes, student performance, and student outcome 
assessment. These data would be used to “educate the committee membership on the education 
climate in Maine.” 

The Systemic Planning and Evaluation Standing Committee was to work with the three other 
major program components of the ME SSI to develop assessment procedures for each of the 
specific program components. Specific areas were to be investigated under each component, 
such as: 




• For Curriculum and Instruction, assess the efficacy of new curriculum frameworks in 
terms of improved instruction in mathematics and science, and improved student 
performance in the Beacon Schools. 

• For Community Integration, annually reassess goals, strategies, and actions for 
community integration programs across the state, with Community Action Teams 
paralleling this process to assess local programs’ impact on needs, to identify 
additional needs, and to develop responses to those needs. 

Responsibilities of the Systemic Planning and Evaluation Standing Committee included 
monitoring progress of all components, recommending changes within the initiative, and 
proposing specific state and local policies, laws, and standards to affect mathematics and science 
education. Initially, the committee would develop preliminary recommendations for the 
Executive Board of Directors to review. In addition, the Committee would develop a plan to 
annually assess and refine the vision, goals, strategies, and action plans of the SSI. A primary 
goal for this committee was to ensure the alignment of all program components. 

The Political Strategy of the Maine SSI 

Those involved in the process of planning and implementing Maine’s proposal for the SSI 
program recall a broad and inclusive process that brought together voices from different parts 
and different sectors of the state, many of which had not worked together around issues of K-12 
education previously. The value of building involvement across the state and across sectors was 
described by one individual: 

The actual grant writing process was just of tremendous value. We brought people 
together that had never been together before. And it opened the door for people to talk to 
people. It made a bunch of connections. Then the formation of the committees and the 
committees’ work, followed by the Beacon Site grant process, which each of these 
processes you know, what was really going on there was that you were bringing groups 
of people together that had never been together before, and talking about what was 
important about math and science education in Maine for all students, and where we 
were and what could be done, and there was just such enormous leverage in that, 
because that is a network that almost explodes out geometrically .. . Those debates [were] 
so interesting, useful, at times heated but they were genuine and built a network across 
the state that had never existed before, and allowed people to contact people, and I 
regularly get calls from educators, sometimes that I don 7 know . . . That never happened 
prior to the SSI. (SSI component leader) 

The SSI built its major components not only around identified needs in the state, but also around 
the interests of several key players. The State Department of Education was a key organization 
involved in the Curriculum and Instruction Standing Committee, chaired initially by the State 
Mathematics Consultant. Institutions of higher education were in the lead for the Professional 
Preparation and Development Standing Committee, chaired initially by the Dean of the College 
of Sciences, University of Maine. Business, research, and community partners were mainly 
involved in the Community Integration and Systemic Planning and Evaluation Standing 
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Committees, respectively chaired by a Senior Research Scientist at the Bigelow Laboratory for 
Ocean Sciences, and a Senior Manager for Inventory Planning and Strategy at L.L. Bean, Inc. 

The development of the plan to involve individuals and organizations from many sectors 
throughout the state was made with considerable forethought. As one Co-PI said, “We 
tried to appease a lot of people sitting around the table that were really, really interested 
and not just there to be there. They were there to work on it.” Consequently, the SSI plan 
included some activities with roles created for supporters from different sectors. These 
activities offered a way for supporters to identify with a specific piece of the initiative that 
directly addressed their interests and concerns. However, some of these activities were 
intentionally isolated within the initiative’s plan, so that if one or more did not succeed or 
could not be sustained in the long-term, the overall initiative could still go forward. 

A Plan for Implementing the Maine SSI 

The vision for the ME SSI articulated an improved “model of the process of education.” The 
implementation of the vision relied heavily on developing models that could be tested, refined, 
demonstrated, and disseminated throughout the state. The planned models recognized the 
complex interactions locally and statewide among different components of society and the 
education system. Such components included community awareness and support, teacher 
preparation and professional development, guiding curriculum standards, effective 
implementation of standards-based curriculum, improved resources for teaching, and assessment 
and evaluation to inform ongoing planning. The basis for this strategy of implementation was 
the recognition of local control in Maine. Policy documents such as curriculum frameworks, 
standards, and the Maine Educational Assessment, although issued from the Maine Department 
of Education, carried few stakes for districts and were to be used primarily as models or guides 
for developing local curricula and instructional improvement. Multiple local models for 
innovation and reform were sought in the SSI plan so that local communities could respond to 
the state policy documents, but also to their own local needs, utilizing their strongest local 
capacities, and involving the local human and financial resources as they saw fit. By developing 
several local models with a regional basis, it was expected that other communities would be able 
to emulate or adapt aspects of one or more models to address mathematics and science education 
reform in their schools. 

The planning strategy for the statewide initiative was to be mirrored in local contexts. MMSA 
implemented its plan for Beacon Schools, the Beacon College, and the Community Action 
Teams in much the same way that the SSI had been planned, by crafting a request for proposals 
that required the convening of a broad-based group of stakeholders locally to conduct a self- 
assessment of needs and capacities and to create a plan for reform. Local decision-making 
continued to be valued. For example, in the case of the Beacon Schools, although MMSA 
recruited and accepted applications for the Mathematics and Science Facilitators, the selected 
Beacon Schools ultimately interviewed and hired their own Facilitators from among this group 
of applicants. 

The addition of two full-time support staff and the opportunities for professional growth among 
all science and mathematics teachers as a Beacon School was a strong incentive for participation 
in the ME SSI. The ME SSI also drew upon “An Act Relating to Restructuring the Public 
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Schools” as an incentive for statewide policy change, especially as revised frameworks, 
standards, and assessments were in development and school districts would need assistance to 
respond to the new policies. 

Scaling-up and sustaining systemic reform in Maine depended on the development of high 
quality policy documents, the development of the Beacon Schools as model schools with strong 
community support from their Community Action Teams, the development of infrastructure for 
professional development through the MMSA sponsored academies and the Beacon College, and 
the use of Systemic Planning and Evaluation as a tool to understand what activities were creating 
and leveraging reform. A leader of the SSI described one view of the intended plan for scale-up 
and sustainability: 

When I think of the SSI I think of the Beacon Schools and that component. That was to 
me [was] the most important one, and the rest ... were in support of that to me. ... I 
mean you have to have ... community support in order for something like this to work, so 
you have to do that. The idea of the institutes that was if you were a faculty are going to 
be teaching in these schools you have to help them learn this material, learn this 
approach and it was a way of using the people who we would be developing in the 
Beacon Schools to spread the word through these institutes. ... The policy level ... if you 
don ’t have your policy makers as such in line with what you ’re trying to do in like a 
Beacon School setting, . . . ultimately the change won ’t be lasting. 

. . . We used the word “ Beacon ” School . . . wonderful places where something really 
exciting is going on, and so that the school district next door would say, “You know 
something interesting going on there, let’s go over there and let’s talk to them, ” or more 
than that that the people in the Beacon Schools, one of the charges was to go out and 
spread the word through the various mechanism that were set up, such as the institutes 
during the summer. The Facilitators, one of the components as I recall, was built into the 
Facilitators’ job, was to try to communicate to other schools. 

Another leader described the plan for scale-up and sustainability with greater attention to 
evaluation and planning: 

The Beacon Schools were intended to be able to try different things . . . real living 
laboratories, action learning kinds of environments. ... Try something and see what 
happens. And the intent would be to really measure what happens, so that we could see 
that we were making progress and that we were not making progress, so the notion of the 
living laboratory, / think it made a lot of sense. 

...I think that the view was that while we had some perspectives on what was required, 
they were not widely shared. So we needed to have a mechanism to either prove them or 
disprove them, then have the results become part of the policy, and have ways of 
engaging the community so that they understood and bought into the changes as well. 
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Maine Statewide Systemic Initiative 
1992-1997 

The beginning of the ME SSI proceeded on several fronts. MMSA was established as a non- 
profit 501(C)3 organization and housed in the same building as the Department of Education. 

An Executive Director of MMSA was appointed to oversee organization and implementation of 
the SSI plan. Among the first tasks of MMSA were to convene its Standing Committees; to 
write its first policy document, Maine ’s Curriculum Framework for Mathematics and Science', to 
develop RFPs for the Beacon Schools and develop the Beacon College; and to write job 
descriptions for the Mathematics and Science Facilitators and begin recruiting applicants for 
those positions. 

Developing Supportive Policies 

Participation in the development of the Framework was a notable early success for the SSI. The 
Framework was viewed as a companion to Maine’s existing Common Core of Learning specific 
to mathematics and science, but was also meant to introduce ideas from the national standards 
movement into mathematics and science education in Maine. MMSA took the lead in 
constructing the Framework, seeking to craft a guide for local curriculum development. One Co- 
PI recalled: 

We had a curriculum framework grant from the U.S. Department of Education and that 
started us down the road to the Learning Results, and we had a lovely curriculum 
framework and we have a lovely curriculum framework in math and science, a combined 
piece, although the content is separate. It’s more like the National Science Education 
Standards in that there are professional development and system standards as well as the 
content standards for the two areas. But that moved us to begin to gain a consensus 
around what are the important things for kids to know and be able to do in science and 
math. 

The Framework was reviewed favorably by state and national review panels. The Framework 
and MMSA gained both credibility and leverage through these positive external reviews. The 
Framework became a broadly known and respected guide for local curricula and professional 
development, providing a common guidance and some quality control for reform across the state. 
The development of the Framework also put MMSA in a favorable position for continued policy 
work on the Maine Learning Results, a curriculum-wide standards document, and the ongoing 
revision of the MEA, including changes to align the statewide assessments to the Framework and 
Learning Results. 

In its final annual report to NSF, the ME SSI noted its central role in decision making for the 
Learning Results Bill. Staff provided testimony before legislative sessions of the Joint 
Committee on Education and the Governor’s Productivity Realization Task Force. The Learning 
Results Bill identified mathematics and science/technology as two of eight content areas in 
which Content Standards and Performance Indicators would be written. A Critical Review 
Committee ultimately considered the work of the Task Force on Learning Results in light of 
suggested changes and published the revised content standards and performance standards for 
statewide review before the enactment of the Learning Results. The Critical Review Committee 
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was advised by the Standards Committee of the Department of Education, on which ME SSI 
members also served. The Learning Results, ultimately approved in 1997, communicate guiding 
principles, content standards, performance indicators, and a foundation for the Maine 
Educational Assessment system. One SSI leader described the involvement of MMSA in the 
Learning Results and their importance in the state: 

I think that the Learning Results have been just a tremendous thing. I think the Alliance 
had a big part of that. A lot of people claim credit for it, as they should, because it was a 
statewide effort in involving probably thousands of people. Having said that, I think the 
effort of the Alliance and the effort of the Alliance ’s portion of the math and science 
reform opened people’s eyes and created an environment where the Learning Results 
could come forward. I think it created a demand for Learning Results and certain 
dissatisfaction with the present that allowed people to say, “This is just not good enough. 
Business as usual or education the way we used to do it isn ’t going to work. So you ’ ve 
got to do something different. ” 

By 1997, nearly one-third of Maine’s schools were using the Framework and the Learning 
Results to align their curriculum. MMSA provided additional assistance to many of those 
schools. 

In its first five years MMSA, its Facilitators, and other affiliates participated on many other 
policy committees and task forces in the state, including the Comprehensive Assessment System 
advisory board, Maine Internet Education Consortium, Maine Educational Assessment 
development teams, State Technology Task Force, Tech Prep Executive Committee, and the 
Maine Leadership Consortium. 

MMSA accomplished a great deal in term of forging political alliances as well. Leaders of the 
ME SSI identified, in particular, the support of the Commissioner of Education, the President of 
Bowdoin College, who served as Chair of the Executive Board, and several key legislators as 
critical to the establishment and progress of the SSI. Maine, like many SSI states, experienced 
changes in the Governor’s office and concurrent turnover in offices of the Department of 
Education, particularly the Commissioner’s office. Although leaders of the SSI offered 
somewhat different perspectives on the extent to which changes in state level leadership affected 
the SSI, all of the interviewed leaders credited the broad-based support structure and leadership 
of MMSA for the endurance of the SSI through those changes. Those political alliances were 
recalled by two Co-PIs as especially important in establishing and maintaining the SSI: 

The chair of the executive committee was [the] President of Bowdoin College. So he had 
good connections, good vision, he had good communications skills, good bring-people- 
together, collaboration skills. And he could talk to the governor without a problem. He 
could bring the Alliance together without a problem. He could get things from people on 
the board fairly easily. So he provided a lot of leadership. 

We had gathered enough support from the legislature to not really have to fight a major 
battle. I mean, we had a little, to keep that [state] money in there every year. And that 
was a reed pleasant thing that we got that money. And we had a little bit of selling, but 
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not the major battle. ... In other words it seems to be a little bit privileged money, kind of 
set there . . . but didn 7 get attacked. 

Beacon Schools, Facilitators, and the Difficulties of Scaling-up the Reform 

A combination of influences from the NSF review team that advised Maine prior to funding the 
SSI and forces within the state prompted changes to the Beacon School plan. First, the plan to 
identify one Beacon School in the first year as a pilot site was abandoned. All seven Beacon 
Schools were to be identified as early as possible in order for the SSI to reach more schools and 
teachers more quickly. Moreover, the original idea of identifying single schools as Beacon 
Schools had already been modified to allow districts to identify either one school or a collection 
of schools as a Beacon Center. 

Once identified and matched with a Beacon Center, Facilitators were deployed almost 
immediately. MMSA provided support and professional development for the Facilitators, but the 
majority of their time and duties were devoted to the Beacon Centers. 

Some notable successes were achieved in the Beacon Centers. More than 80 percent of the 
mathematics and science teaching faculties of the 38 schools in the seven Beacon Centers were 
directly involved in MMSA professional development. Approximately, 40 percent of those 
teachers reported increased time devoted to mathematics and science instruction; 66 percent 
reported increased collaboration with other teachers. A significantly greater number of students 
graduating from Beacon Center high schools were pursuing post- secondary education in 1995 
(65 percent) compared to 1992 (59 percent). Progress in the Beacon Schools was variable, 
however. Several interviewed leaders indicated that some of the Beacon Schools made little 
progress toward reform, but all seven remained as the ME SSI’s model schools. 

The work of the Beacon Centers and Facilitators was not without its frustrations. Facilitators 
may have faced mixed messages about their priorities, stemming from their responsibilities to 
both MMSA and to their local schools, district, and community. An SSI Co-PI described the 
issue: 



[The Facilitators] were selected as people we expected to be prepared but then once they 
got the job that was a prime responsibility of the Alliance ...to begin to work with them, 
to train them further. ... In some ways there was probably an overly dynamic situation, 
but they, in essence, had two bosses: the Alliance telling them, “We need you to do these 
things, ” and the locals saying, “We need you to do those things. ” So that the facilitators 
had to match them as possible and determine— “which audience do I satisfy first?” 

Moreover, the work of many Facilitators had become complicated by the expansion from single 
Beacon Schools to multiple schools. The work of supporting teachers in multiple schools was 
further complicated by a change in the SSI in response to an NSF site visit midway through the 
initiative. Due to growing concerns about scaling-up the reform beyond the Beacon Schools, the 
SSI initiated its work throughout the state in Beacon Regions. Although the intention of the 
Beacon Schools activity was always to provide infrastructure for supporting reform throughout 
the regions represented by the Beacon Schools, the move to Beacon Regions occurred abruptly 
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and much more as a statewide directive to the Beacon Schools and Facilitators than as a natural 
evolution as had been envisioned. A Co-PI described the decision: 

The facilitators ... were told that they needed to get out of their schools and work in their 
regions more. ... I think they became close to 50/50 in their school and their region. So 
they were now torn yet again, because they have the Alliance to listen to and their locals 
to listen to, and there’ re supposed to be in their region 50% of the time. ... Maybe we 
weren ’t as clear and maybe we didn 7 even know when we put the plan together that that 
would happen. I think we knew it, but we didn 7 think through it. But we knew that those 
people had to get out of their schools, but we didn 7 think that, “Oh yeah, that’s yet then 
another set of responsibilities that is going to be a change for them. ” 

. . . In some places it makes sense, people could deed with it, other places that couldn 7 
have. ... Some of them saw this as a natural progression others said, “It’s too much. I 
can 7 do what I’m doing now. ” 

The Beacon Schools undoubtedly achieved some successes in expanding professional 
opportunities and connections for teachers and changing instruction in the pockets of 
schools they reached most intensively, and more than 100 non-Beacon Schools had 
contacted MMSA or a Beacon School for assistance by 1996. The Beacon Region idea, 
however, did not build a clear infrastructure or create the capacity necessary to foster 
statewide systemic reform in mathematics and science. One Co-PI reflected on this 
aspect of the ME SSI: 

Seven Beacon Schools couldn 7 disseminate to a whole state, even though they were 
strategically placed. ... We tried to have a representative from seven ... different 
locations. . . . Beacon Schools had to have a location in their favor; they had to have a 
reflective plan strategy; they had to be able to look at a long plan; they had to look at 
how they would support it, etc., etc., all of those things. But to only have seven was not 
enough, and I think that became pretty obvious fairly early to us, and so what we tried to 
do is to have things happen like in the summer time or on vacations that were either led 
by the facilitators or were for them but we added other people. And we started with those 
62 or 63 other schools that had applied [to be Beacon Schools], but we also had people 
ask us, which was kind of pleasant, that schools asked us to ... how could they be 
involved or how could they get some of the training or how could they work with us in 
some way? 

Beacon College and Professional Development 

The idea of selecting a single Beacon College from among applicants was also abandoned early 
in the project, in favor of a more inclusive plan. Rather than create an RFP and invite 
applications, several public and private institutions of higher education in the state instead 
developed an idea for a multi-campus, virtual Beacon College. The idea gained sufficient 
momentum to result in a proposal from Maine to NSF’s Collaboratives for Excellence in Teacher 
Preparation program. This proposal, however, was not funded and many of the SSI leaders saw 
development of the proposal as the final act of the Beacon College within the funding period of 
the SSI: 
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We just were never able to pull that one off. It was actually, as I recall, it was another 
proposal that went in for funding of the Beacon College later, that just went right down. 

I don 7 think we ever had a good vision on that one. There were some ideas; it just didn 7 
gel. It was trying to do something I always felt a little odd about the way that was, the 
way people are trying to pull that together, people from various colleges and universities 
in this sort of virtual college. And at any rate, so that one just never got off the ground as 
best as I know. (SSI developer) 

Although the Beacon College was never clearly established as envisioned, several institutions of 
higher education held summer professional development academies as a part of the SSL The 
academies served teachers from the Beacon Schools as well as other schools throughout the 
state. Five of the 6 University of Maine campuses and three of the state’s private liberal arts 
colleges held extensive professional development institutes, which took on the name Academies, 
for teachers of mathematics or science in elementary, middle, and high schools, often with an 
integrated mathematics and science component. One SSI leader reflected on the importance of 
the Academies for reforming instruction, and the extent of their impact: 

I think the teacher Academies brought a level of expertise to math and science education 
over time. And I think those Academies did a couple of things. . . . They opened the door 
for a new way of teaching. But cdso I think they invigorated teachers that were already 
good but in a rut. I think that made a big difference. I think it elevated the status of math 
and science education and educators. It made them feel important. It made them be 
recognized for their expertise, and in the communities ’ eyes as well. Not that other 
educators aren 7 important. It takes everybody, but I think we were able to have an 
impact on those people. 

The Academies were also intended “to bring higher ed into the mix a little stronger. Because 
we’re giving them money, we could maybe change pre-service, maybe change their in-service 
offerings and maybe even have them think about their entire undergraduate program.” Those 
types of changes may have been embodied in the Beacon College proposal for a Collaborative 
for Excellence in Teacher Preparation, but without additional support for that proposal, such 
changes were not realized. 

The SSI was also able to leverage other professional development opportunities through MMSA. 
PRISM conferences continued with support from MMSA and the Department of Education and 
remained well attended. MMSA partnered with the Department of Education to conduct 
seminars across the state on implementing the Framework. Six schools were specifically 
selected as Curriculum Framework Implementation sites by the Department of Education and 
supported by MMSA professional development opportunities. Also, the Department established 
nine Teacher Leadership Consortia with a small pool of money to stimulate communication 
across schools regionally in Maine and to further develop leadership in mathematics and science 
reform among teachers. 
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By 1996, the Academies had served 630 participants from 220 schools in 113 school districts. 
The Curriculum Framework Pilot sites served 225 teachers, and PRISM attracted 1,000-1,600 
teachers annually. 

Getting Communities on Board 

Development of Community Action Teams proceeded as planned in the Beacon Centers. 

Midway into the SSI, the leadership of the Community Integration Standing Committee changed 
as the original leader left the state and another business representative took over leadership of the 
committee. The focus of the Community Action Teams did not change considerably with the 
change in leadership, but the structure of the activity did change somewhat at about that time. At 
the request of the new leader, MMSA hired a communication specialist to assist with community 
integration. The communication specialist brought a public relations and marketing perspective 
to the community integration work. 

The nature and goals of the community integration work were described as: 

One [goal] was that we wanted to simply publicize and educate the Maine public about 
the importance of math and science education and the current state of it, ugly though it 
might be. So we attempted to take it on anyway we could to get the word out. We did 
math and science workshops at McDonald’ s and all kinds ofhokey things to get the word 
out that here ’s what you should expect for good math and science education. This is kind 
of what it looks like. This is what you may be experiencing. This is what you can do 
about it. You can call us and you can talk to your school people. You can do this and 
you can do that. So we proceeded to go out and teach these people how to do these math 
and science workshops in the evening at a community level, where parents, students and 
community people would come in and experience hands-on, the educators would use . . . 
hands-on math in a fun way to make math real, which was the second part of our 
community [integration]. Maine has a big aspiration problem. As a poor state, a lot of 
people didn 7 do well in school. So they reflect on the school through their own 
experience, and therefore math and science was in the couple of areas that they struggled 
with most, was most uncomfortable with. So we had to take that on as well as the best 
that we could. Talk about students that were under-served because it was about all 
students, not just the best and brightest and particularly all students really, in all parts of 
Maine. (SSI Component Leader) 

Among the Community Integration Standing Committee’s work were collaboration on the 
production of a public television education series called Quest, a CD-ROM for teachers, Family 
Math and Science handbooks and events, and parent and community volunteer programs. 
Partnership with National Semiconductor facilitated publication of these educational materials 
and SSI members gave presentations about using them to community leaders. The SSI hosted 
two statewide community integration conferences and presented on community integration issues 
at another three conferences for education administrators. 

The impact of the Community Action Teams was difficult for the SSI’s leaders to describe. 
Nearly all expressed a sense of the importance of the work undertaken by the Community 
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Integration Standing Committee, but were unclear about its successes and failures in supporting 
systemic reform: 

One area that we ... struggled with ...is the evaluation and assessment of the 
effectiveness of community activities. ...It seemed logical and intrinsically good that that 
would over time have an impact but I wasn ’t able to say that . . . community activities, 
while appearing to be inherently good on the surface, ultimately changed any of the MEA 
results one point one way or the other. We were never able to get there. We were a third 
order effect. I think part of the work that we did allowed and greased the skids for the 
Learning Results. I think it opened to door for people being willing to talk to educators 
about the value of education. I think it did a lot of great things but it was never able to 
get that far. . . . Because it was so compelling maybe [we] didn 7 spend enough time 
thinking about: What do we really want to get out of this? What is our strategic goal? 
And how does that translate into tactical goals? And how do we assess whether we ’ve 
succeeded or not? (SSI Component Leader) 

Managing a Complex Initiative through Systemic Planning and Evaluation 

The Systemic Planning and Evaluation Standing Committee began its work with the intention of 
creating a model process initially. The SSI intended to spend its first year developing an 
“effective assessment framework and conducting a thorough evaluation of Maine’s existing 
educational process and specific elements of the restructuring efforts.” The rationale behind this 
approach was that the SSI should target areas where change is needed most and thus they must 
first identify those areas. The accelerated start-up of the Beacon School component may have 
prevented this approach from being fully utilized. 

The committee did proceed with its evaluation and planning work, however. An external 
evaluator was contracted through the Center for Research and Evaluation at the University of 
Maine. 

Systemic planning occurred in a fashion somewhat similar to the arrangement of NSF’s 
cooperative agreement. For example, MMSA initiated competitive processes through RFPs for 
Beacon Schools, Academies, and other programs. As one leader described it: 

The intent was to have a competitive process for the money that we had. That part of 
what the winner would get not only the money, but us. So we became participants in 
their process and we became, in essence, their [formative] evaluators, and that’s how the 
systemic planning process was to occur. 

MMSA’s Systemic Planning and Evaluation also provided formative evaluation for the other 
components and committees of the SSI. Across components, successes were mixed: 

I think what we were trying to do more than anything was build [systemic planning and 
evaluation ] into each of the components. And so what we tried to do was engage each of 
the groups in really understanding: What are we trying to do here? ... What’s really the 
goal? If we want to have a meeting that we feel good about, we can do that. We can do 
that, but are we trying to go beyond that? And let’s be very clear about what the goal is. 



18 




Do we want people to become aware of math education reform as a need? Do we want 
them to understand the state of math education in their community? And in terms of that, 
do we want them to understand both the strengths and weaknesses, how are their kids 
doing? What are some of the complexities? There’s lots of questions like that you may 
want a community to become aware of and we were just never very clear about those 
things, so we ’d have these meetings but we just couldn ’t tell that we were making 
headway. And that’s what we were trying to do — engage the groups in self-evaluation. 
And I would say that in the case of the Beacon Schools that actually worked pretty well. 

... They really worked at trying to understand, trying to be a little bit cold-hearted about, 
“ Are we making progress or not?” I felt in the case of the Beacon College and the 
Community [ Integration ] parts that we didn 7 really engage it that sort of really clear- 
headed view of “ What it is we ’re trying to accomplish ? ” and whether or not we ’re 
making progress. (SSI Component Leader) 

The Place of Equity 

Maine’s original focus on equity targeted female students and non-college bound students. Clear 
gaps on MEA results were presented in the original proposal for these groups versus their peers. 
In the case of females, a widening gap, especially in science achievement, across grade levels 
was a primary concern. The gap in achievement scores in mathematics and science for non- 
college bound students versus students enrolled in college preparatory programs was 
considerable. The SSI targeted equity concerns in instructional practice and potential biases in 
assessment practices. It also instituted the apprenticeship programs through its Community 
Integration Standing Committee’s work to target underserved students in mathematics and 
science. 

By 1996, the achievement gap by gender was narrowing statewide. Despite the work that the 
SSI had conducted related to this issue, no clear research-based link was drawn to support 
attribution to that work. The apprenticeship program had involved more than 400 students 
statewide. Of apprenticeship participants, 100 percent had begun post-secondary education and 
95 percent were majoring in science, mathematics, or engineering. It was not clear that the 
apprenticeship program reached the non-college bound student population in particular, but it did 
enroll more females than males and minority students at a rate twice their representation in the 
state population. 

The equity focus took a decided turn at the midpoint of the initiative. Following a site visit, NSF 
prompted MMSA to identify a specific minority group for its equity focus. The largest minority 
group in Maine is Native American. Consequently, MMSA engaged representatives of five 
Native American communities in Maine with efforts to improve teaching and learning 
opportunities in their schools. MMSA formed a collaboration among these five communities and 
the University of Maine’s Wabanaki Center, supporting programs such as EQUAFS to provide 
professional development for teachers on equity issues. In 1995-96 collaborative work among 
MMSA, the Native American communities, Maine Indian Education Schools, and the Wabanaki 
Center continued to grow. MEA scores of Native American students in both mathematics and 
science at both the 4 th and 8 th grade levels were considerably higher in 1996 compared to 1992. 
Again, however, a clear link to the work of MMSA was not established. 
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Summarizing Accomplishments and Challenges: 
Sustaining Systemic Reform in Maine 



The ME SSI affords an example of an SSI that put interventions into place early in the initiative 
that were not entirely matched with political realities that emerged in the state and in the SSI 
program. Sustaining systemic reform in Maine involved revamping their strategies in light of the 
political successes and failures of the initiative. In particular, MMSA had to be alert to 
opportunities that existed when the SSI funding from NSF came to completion. The nature of 
those opportunities and determinations about the successes and shortcomings of the SSI as 
MMSA contemplated its future led to decisions that produced a continuation of mathematics and 
science reform in Maine, but with a substantially different approach. 

The SSI in Maine was most visible in its creation of Beacon Schools, which were intended to be 
“living laboratories” for creating and demonstrating standards-based instruction in mathematics 
and science education. The Beacon Schools were selected from different regions of the state, 
representing different levels of capacity and progress toward a standards-based system. Pushed 
by forces inside and outside the state, MMSA required the Beacon Schools and Facilitators to 
expand their services to support systemic reform throughout large regions of the state, in many 
cases before the schools and Facilitators were ready to do so. The plan for scale-up was not well 
articulated, and the capacity to accomplish the desired scale-up was not in place in many of the 
Beacon Schools. 

The Beacon College component of the SSI plan was quickly identified by most interviewees as a 
clear shortcoming of the initiative. Despite the fact that several participating institutions of 
higher education had developed a proposal to NSF’s Collaboratives for Excellence in Teacher 
Preparation program to support the Beacon College, the proposal was not funded and the Beacon 
College idea lost steam. Many of the institutions of higher education remained active in other 
areas of the initiative, particularly in providing summer professional development Academies for 
teachers, but systemic change of teacher preparation programs, professional development 
offerings, and undergraduate programs in mathematics and science were far from realized. 

Although many saw the Beacon School/Region component of the initiative as the centerpiece of 
the SSI in Maine, MMSA was also involved in policy work. MMSA was a key player in 
developing Maine’s Curriculum Framework and Fearning Results policy documents, and 
revising the long-standing statewide assessment, the Maine Educational Assessment, in support 
of the framework and the standards of the Fearning Results. MMSA shared staff and leadership 
with the Department of Education, particularly in the persons of the science and mathematics 
supervisors for the state. MMSA was an independent organization, however, and brought 
connections to and resources from business, higher education, and local communities that the 
Department of Education was not in as good a position to include in policy development work 
for frameworks, standards, and assessments. MMSA made itself an invaluable partner to the 
Department of Education. 

It was a nearly devastating blow to MMSA when Maine was not included among the SSIs 
funded by NSF for continuation in Phase II. Discussions ensued regarding whether MMSA 
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should disband altogether. However, a private foundation made an offer to provide continuation 
money to MMSA in gradually decreasing amounts over three years while the initiative 
regrouped. However, the funds would not approach what continuation funding from NSF would 
have been. Even with this offer, the SSI leadership was not immediately convinced it should 
continue: 

The Executive Committee group convened, and said, “Should we just sort of put our stake 
in the ground and say ‘We’re done’ and celebrate?” We did have a celebration anyway 
that just acknowledged everyone ’s work and everything that was going on. We invited 
everyone who was involved. And we gave out lots of awards and things. But inside the 
Alliance it was, “Should we really go forward, or should we end it?” And one of the 
things that I said to them was that I don ’t want to be the group that is just going to sort of 
peter out, you know, that each year something was going to get tighter and tighter, and 
. . . that essentially we should just say, “Hey, we ’re done. We did all that we could and 
we ’re really pleased with what we did. ” Or we should say, “This group, the 
Mathematics and Science Alliance has value. It’s adding value to the state and it’s a 
good contribution and we should keep it. ” And on that basis we could go forward in a 
positive way. And the board decided that there was absolutely a need and that there was 
important value. (SSI Project Director) 

Having reaffirmed commitment to the Alliance from within, the Executive Director knew 
maintaining strong political support was also critical to sustaining the reform: 

I went to talk to the Department, the Commissioner at the time, and said basically the 
same thing, “I don 't think it is worth the state having this group that just sort of peters 
out at the end of the funding. It would be much better to either end it or ... make a 
commitment to it. ” And he said, “Well, my commitment is to it. ” 

When MMSA decided to continue, it was still faced with very difficult choices about what 
components of the initiative should remain. MMSA had also to begin planning immediately for 
self-sustaining support. The Beacon concept was among the first to go. The expense alone may 
have kept the Beacon Sites from being continued, but the lack of a mechanism for the Beacon 
Sites to scale up reform to the whole state proved just as important in the decision to conclude 
this component of the initiative. 

Although the Beacon concept was abandoned, a great deal of important work had been 
accomplished as part of its development: the work in Beacon Sites established MMSA’s local 
visibility and credibility for working with districts; the work on Framework , Learning Results , 
and MEA assessments deepened the partnership with the Department of Education; and the work 
of developing proposals for funding from NSF developed MMSA as an organization that could 
write and manage grants. Perhaps most importantly in Maine, MMSA was itself established as 
an independent, non-partisan entity. At this critical juncture, MMSA did not depend on a parent 
organization to dictate whether or not it could continue. All of these forces had come together to 
provide a basis on which to continue MMSA as a catalyst for reform and an important support 
organization for schools and districts. The ongoing work of MMSA, the Executive Director said, 
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is aligned around four core ideas: “solid mathematics and science content knowledge; decisions 
. . . based on data; high quality professional development and instruction;. . . and . . . equity.” 

MMSA has shifted its focus and diversified its funding base in recent years with ongoing fiscal 
support from the legislature and Department of Education, as well as grant and program support 
from public and private foundations, including new and ongoing programs of NSF. It continues 
to partner with the Department of Education in policy and support work in the state. It has also 
become an umbrella organization for a number of mathematics and science projects and 
programs in the state. MMSA’s connections to schools and districts, its history as a support 
partner for districts looking to implement the Learning Results , coupled with its grant writing 
capabilities have also positioned MMSA to lead improvement efforts such as the Maine Local 
Systemic Change through Teacher Enhancement (LSC) project involving a collection of schools 
and districts across the state implementing standards-based instructional materials in 
mathematics with substantial professional development support for teachers. The current work 
of MMSA was summarized in this way: 

We align ourselves around three goals, what I call goal areas. And the first one is school 
and school district reform in mathematics and science ed. . . . That really is the systemic 
reform effort. That we still support schools in looking broadly at cdl parts of the system, 
community, administration, and all the way up to the teachers as to how to make the 
change. And our LSC would fit under that goal. We have one that is called full 
implementation of high quality, standards-based mathematics and science, and that’s the 
curriculum focus. You know, the LSC creeps into that too, but it’s a little bit broader. So 
that’s supporting schools to use particular standards-based materials. The last one is 
broader, which is to increase the number of qualified teachers of mathematics and 
science. ... That’s our general professional development and higher ed links. (SSI 
Project Director) 

MMSA’s connections to higher education in the state, despite the failure of the Beacon College 
component to obtain independent funding early in the life of the SSI, have come together years 
later. MMSA conducted a study of the state’s mathematics and science teaching force that 
demonstrated a serious impending teacher shortage. In response to findings of the study, MMSA 
partnered with the University of Maine, the University of Southern Maine, and the University of 
Maine at Farmington to submit a new proposal to NSF’s Collaboratives for Excellence in 
Teacher Preparation program. The proposal was funded to create the Maine Mathematics- 
Science Teacher Excellence Collaborative. 

Finally, MMSA has established a strong dedication to promoting teacher leadership in 
mathematics and science in Maine. In earlier work with the Facilitators and the Teacher 
Leadership Consortia, MMSA had conducted leadership work, but has learned from those 
experiences and retooled its leadership training and expectations. MMSA now supports several 
specialized leadership experiences for teachers and provides logistical support for teacher-led 
professional development, rather than expecting teachers with limited time to develop broad and 
deep expertise and arrange for the professional development of their peers. Teacher leaders 
today specialize in areas such as content and assessment, and may be called on to apply that 
expertise to school, district, and state needs as well as to offer leadership in MMSA projects. 
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Graduated leadership training is offered so that teachers may develop leadership skills from a 
novice level to a very high level. 



Factors that Shaped the Maine SSI Story 

The ME SSI did not accomplish statewide systemic reform in mathematics and science. Its 
accomplishments toward systemic reform, however, should not be minimized. Through its many 
coordinated activities the SSI contributed to creating a supportive state policy structure for 
supporting standards-based reform in science and mathematics. It helped a number of schools 
and school districts begin to address standards-based reform through technical assistance for 
planning improvements, local policy alignment, professional development of teachers, and 
engagement of the community in understanding and supporting standards-based reform. It also 
clearly accomplished the institutionalization of MMSA as a strong, independent voice for 
mathematics and science education in the state. MMSA has evolved primarily into a technical 
assistance center for districts to respond to state policies, internal reform efforts, and external 
reform opportunities. MMSA is also a recognized advocate for mathematics and science 
education at the state level. Districts, the Department of Education, institutions of higher 
education, and businesses and communities have been able to utilize the recognizability and 
credibility of MMSA within the state and nationally to support many mathematics and science 
education reform efforts. 

A number of lessons have emerged from the Maine SSI that may be applicable across states and 
in other statewide reform efforts: 

> Leadership of many kinds and at many levels is critical for reform. 

The support of many leaders in Maine contributed to the successes of the SSI. In particular, 
broad-based leadership including education, higher education, government, business and 
research, and communities were seen as key factors for the SSI. Turnover in leadership at all 
levels did not have a great impact on the SSI, largely because so many leaders in the group 
remained when any one turned over. Also, it was critical for Maine to have a group of leaders 
that collectively have the ability to lobby important decision-makers in the state, to convene 
advisory and working groups for the reform effort, and to conduct the day-to-day business of 
reform. No one person had all of those abilities, and it may be that if one person had been relied 
on for all of that, a change in leadership would have resulted in substantial problems in 
continuing the reform. 

> Establishing credibility early in the reform opens many doors. 

ME SSI began its work on many activities simultaneously. However, the development, 
publications, and favorable external reviews of Maine \s Curriculum Framework for Mathematics 
and Science was a seminal activity for the SSI. The Framework established MMSA as an 
organization that could engage in policy work leading to a great deal of ongoing involvement in 
developing the Learning Results and revision of the state assessment. Capitalizing on the 
Framework in its professional development and community engagement work, MMSA created a 
demand for its services. It positioned itself as an ideal partner for districts seeking help in 
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aligning curriculum to the Framework or Learning Results and initiating programs to improve 
instruction, such as the LSC. 



> “Model” approaches require careful consideration of what makes a model and what 
makes a model useful. 

The Beacon School approach to systemic reform was compelling, but ultimately lacked a few 
critical ingredients. First, despite the recognition of MMSA that evaluation results would be the 
clearest indicator that the model schools were engaged in reform worth emulating, the collection 
of data locally to link the ME SSI activities to the desired changes in instruction and student 
achievement was frequently a roadblock. Evaluation results never clearly demonstrated an 
impact of particular activities on particular outcomes. Many schools and districts contacted 
MMSA and the Beacon Schools for assistance in any case, and the ME SSI and NSF 
increasingly pushed the Beacon Schools and Facilitators to serve more schools in their regions, 
but a clear structure for utilizing the Beacon Schools as models for systemic improvements was 
not established. Ultimately, scale-up of systemic reform according to the Beacon School model 
could not be achieved. 

> Establishing an independent organization as a center for systemic reform has definite 
advantages. 

The establishment of MMSA as the agent and home of the ME SSI afforded a number of 
advantages in the state. First, as a local control state, much of the activity of MMSA could be 
seen as coming from an external source of assistance to the locally-controlled school districts, 
rather than as state imposed recommendations or mandates. Second, convening a broad-based 
group to support mathematics and science reform in Maine was made easier by creating the new 
“turf’ of MMSA with a shared sense of ownership, rather than housing the reform in an existing 
agency where some important stakeholders would hold more sway than others and some might 
not feel comfortable participating. Third, having established its credibility, MMSA gave a voice 
to science and mathematics education that did not exist in the state prior to 1990. A multi- 
million dollar non-profit organization carries considerable weight in government, business, and 
education circles. Finally, the ebbs and flows of state politics have had relatively little impact on 
MMSA. In particular, despite downsizing its professional staff since the end of NSF’s funding, 
MMSA still employs many more experts devoted to science and mathematics education in the 
state than the Department of Education had ever employed. 
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Appendix I 



Case Report 

Michigan Statewide Systemic Initiative 




Introduction 



In 1992, the National Science Foundation awarded the Michigan Department of Education a 
five-year, $10 mi llion Statewide Systemic Initiative grant. Like other statewide systemic 
initiatives, Michigan’s goals were ambitious: to transform the ways that mathematics and science 
were “taught, learned, assessed, and perceived,” and ensure high quality mathematics and 
science education for all students in grades K-12 and beyond. 

Strategies for achieving these goals were multifaceted. The Michigan Statewide Systemic 
Initiative (MSSI) planned to build consensus around a vision for reform; promote policy 
alignment; create working models of change; redesign teacher education programs; and build 
state and local capacity for implementing reform. In the context of existing systemic reform 
efforts in Michigan, the MSSI sought a distinctive niche, looking to strengthen infrastructure, 
link stakeholders and institutions, and bring coherence to the mathematics and science education 
system. These activities cast the MSSI in the roles of advocate, collaborator, catalyst, facilitator, 
researcher, and disseminator. 

This case study takes a retrospective look at what the MSSI accomplished in its five years, the 
ways in which it achieved these successes, and the barriers that stood in its way. Among the 
documents reviewed for this report are: MSSI proposals submitted to the National Science 
Foundation (NSF) for Phase I and Phase II funding; the MSSI Mid-Point Review and Program 
Effectiveness Reports; and external monitoring and evaluation reports. Telephone interviews 
with six persons who were closely involved with various components of the MSSI supplemented 
the details conveyed in these documents. As respondents reflected on their achievements, as 
well as the challenges they encountered, they shared a more personal view of what it takes to 
change the science and mathematics education system. This case study tells their story: their 
vision for reform, the choices they made, and the footprint they left behind. 



Ten Years Ago: 

The Context for Mathematics and Science Education Reform in Michigan 

The state of Michigan covers 57,000 square miles, running 400 miles north to south and 300 
miles east to west. The state ranks eighth in population in the United States, with over 9 million 
people. In 1990, Michigan’s K-12 education system was serving more than 1.8 million students 
in 550 local school districts. The communities served by these districts are economically and 
geographically diverse, ranging from high density, urban neighborhoods to rural, isolated towns 
in the northern part of the state. Approximately three-quarters of Michigan's students are white; 
18 percent are African-American and 2 percent are Latino. Over 43,000 educators were teaching 
science or mathematics in grades K-12 at the time the MSSI was funded. 

The Policy Context 

Ten years ago, Michigan had set the stage for systemic reform in science and mathematics. 
Mathematics educators had developed and pushed for the adoption of “Curriculum Outcomes 
and Essential Goals” by 1989; similar guidelines were adopted for science in 1991. Both the 
science and mathematics “Essential Goals” documents reflected emerging national standards, 
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and provided the basis for revisions to the Michigan Educational Assessment Program 
(MEAP) — the statewide assessment system in science, mathematics, and reading. At the time 
the MSSI was funded in 1992, Michigan was also in the process of developing state standards in 
mathematics and science, based on the “Essential Goals” documents. 

In part, Public Act 25 (P.A. 25), enacted in 1990 by the Michigan legislature, was driving broad 
curricular reform across disciplines. The loss of large numbers of manufacturing jobs in the 
state, and the subsequent changes in the economic landscape, had prompted Michigan’s business 
community to question students’ preparation for the workforce and push the state toward higher 
standards and accountability. P.A. 25 also called for changes in organization, decision-making, 
and accountability, with a redistribution of power within the system — from the state to the 
districts, from the central office to the school building, and from educators to the community. 

While P.A. 25 advocated local flexibility in meeting state guidelines, it also included “ground 
rules” to ensure that schools and districts wove common “threads into their fabric of operations.” 
Accountability was central. Annual performance reports for each school were to include MEAP 
scores, retention rates, and data on suspension, as well as progress toward meeting goals 
described in School Improvement Plans; school accreditation was contingent upon schools 
fulfilling these commitments. If P.A. 25 was designed to bring accountability to the policy 
forefront, it has surely had that effect on school personnel. Said one former MSSI staff person: 
“The pressure [of the MEAP] is immense. Public attention is great. Principals and others live in 
fear of bad scores being broadcast on TV.” The most “at-risk” districts were those most 
“terrified by the MEAP and accreditation process.” 

Still, P.A. 25 held potential for unifying and coordinating the education system. It also set a 
friendly stage for the MSSI by providing an opportunity for reform leaders to push for 
improvements in science and mathematics education within state-mandated structures and 
guidelines. Said one person, “Things happening coincidentally. The state law requiring School 
Improvement Plans and annual reports to the community played into our favor. It allowed us to 
encourage schools to build mathematics and science into their Plans.” In short, as the MSSI was 
taking shape, there was solid evidence that the climate in the state supported higher standards, 
greater coherence among its policies, and stronger accountability. At the same time, resistance 
by schools and districts to high level policy mandates and accountability measures continued to 
feed a long-standing tension between state and local control. 

Bountiful Resources, Fragmented Programs 

At the time the MSSI was funded, Michigan had much to offer in support of science and 
mathematics education. For example, in 1988-89, the state legislature had initiated the 
Mathematics and Science Challenge Grant Program, which established regional Mathematics 
and Science Centers throughout the state to work with local school districts to improve science 
and mathematics instruction. Michigan also boasted numerous programs that supported 
curriculum reform and professional development in science and mathematics. Two projects 
partially funded through Eisenhower grants — the Michigan Mathematics In-service Project and 
Making Mathematics Accessible to All — helped acquaint teachers with the state’s standards, and 
supported them in using innovative materials and in working with underrepresented students. 
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In addition, several Michigan universities had received NSF funding for large curriculum 
projects in science and mathematics, including the Connected Mathematics Project. The Science 
Education in Michigan Schools (SEMS+) consortium was creating grade K-12 science modules, 
and was training curriculum development leaders throughout the state in their use. At the same 
time, the Michigan Department of Education was developing interdisciplinary units in science 
with funding from the Kellogg Foundation. Finally, Michigan’s professional organizations — the 
Michigan Council of Teachers of Mathematics (MCTM) and the Michigan Science Teachers 
Association (MSTA) — were active players in professional development, and in curriculum and 
assessment activities. 

While all of these efforts were of value and signaled interest and activity in improving science 
and mathematics education, the overall result was an accumulation of disparate efforts, with no 
unifying vision and no mechanism for coordination or collaboration. Reflecting on the state 
context at the time the MSSI was funded, these two former staff persons characterized the system 
this way: 

Michigan had a rich array of initiatives and resources in science and mathematics but it 
was very fragmented. There was little or no coherence. There was some early movement 
toward standards and frameworks, but it was not an organized system. There were lots of 
activities and lots of duplication of effort, but there was no alignment, and there was very 
little communication across these entities. It was not very sophisticated at all. Like most 
states, Michigan suffered from project-itis. (MSSI Component Coordinator) 

The mean reason we went for the [SSI] grant was to develop coordination, a more 
comprehensive direction for mathematics and science, one that included the 
identification of a framework everyone would buy into, one that would bring mathematics 
and science closer together, to get people talking together in a way that had not occurred 
before. But primarily it was to get some cohesiveness in the direction that we wanted to 
go in mathematics and science. There was not much direction then, just splintered and 
separate strategies for mathematics and science. And there was no strategy that was tied 
to other school improvement projects within the Department of Education. (Former 
MSSI PI) 

Even the Mathematics and Science Centers that provided regional support were “separate 
themselves,” acting as “independent agencies doing their own thing.” The scenario was similar 
in higher education, with a “mishmash” of grants. With the SSI came opportunity. Reflected 
one former MSSI staff person: “I wouldn’t say people were working in opposition, but they were 
working toward an end without any reflection or interaction with others who had similar things 
going on. We saw the SSI as an opportunity to connect all of these efforts.” 
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The Plan for Reform: 

Filling the Gaps, Enabling the System 



Defining the Niche 

In designing a plan for systemic reform, the MSSI took stock of both needs and resources. 

Among the opportunities in Michigan were curricular and assessment policies already 
established or in development, and a regional system for supporting districts in science and 
mathematics. The needs were all too evident. With much activity focused on improving science 
and mathematics, disparities were nonetheless glaring: on the National Assessment of 
Educational Progress (NAEP), African-American 4 th and 8 th graders were more than three times 
as likely to score below the basic achievement level in mathematics than their white peers. In the 
eyes of MSSI staff, improving science and mathematics education for these students represented 
the “greatest challenge” and a “special point of leverage on the system.” 

Within this state context, the MSSI saw clear roles for itself. Strategies for achieving reform 
would bring coherence to the system, and address critical areas currently neglected by other 
forces in the state. For example, with the state continuing to fund the Mathematics and Science 
Centers, and the U. S. Department of Education providing support for the development of 
curriculum frameworks, the MSSI chose to direct their attention elsewhere, viewing these 
existing pieces as “tools” for reform. Said the MSSI evaluator: 

[The MSSI] looked to see where there were gaps. For example, they chose not to focus 
on issues around standards because [ that] was cdready in the works. They cdso decided 
Michigan was too big geographically to provide professional development. We 'd never 
begin to touch the 60,000 teachers.... The early players had a pretty good understanding 
of what it meant to change the system. It didn ’t mean doing more of the same or doing 
everything. It meant trying to find ways to change the system by leverage points. 

While the state provided guidance on curriculum, instruction, and assessment, there was little 
consensus on the overall direction needed to achieve systemic reform in science and mathematics 
education. The MSSI would articulate this vision. Initially, the plan focused heavily on 
professional development around the state’s emerging standards. Citing a need to expand this 
mission and show evidence of commitment by high-level stakeholders, NSF rejected the Cohort I 
proposal submitted by the MSSI. 

In response, reform planners convened over 100 stakeholders in mathematics and science 
education to reflect on the needs and gaps in the system. A draft position paper entitled 
“Scientific Literacy and Mathematical Power for All” helped generate dialogue and build 
consensus. Discussions were designed not only to inform policymakers, district administrators, 
professional development providers, university faculty, and others, but also to engage them in 
decision-making about how to proceed. For example, there was broad consensus among MSSI 
planners that teacher education reform was a pressing need, and there were no statewide efforts 
in place “to take it on.” Remembers one of the MSSI evaluators: 
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The MSSI provided a way to convene people which hadn ’t been happening. There were 
cdl these pieces out there, but no entity to convene the players. So they got people 
together and got them talking in the same ways and the vision emerged from that work. 
And in the process, the MSSI was trying to build the capacity of the various players — not 
just individual capacity, but also organizational and structured capacities that hadn 7 
existed before. 

By design, the “vision” document encompassed broad interests and activities around science and 
mathematics education. Ultimately, the MSSI would submit the position paper for review, 
revision, and adoption by the Governor, the State Board of Education, the legislature, and 
presidents of universities and colleges. Once adopted, the document would guide systemic 
reform in the state, and serve as the official position statement on mathematics and science 
education. 

A Plan for Action 

In the proposal funded by NSF in its second round of awards, the MSSI design sought to 
influence the science and mathematics education system at the highest policy levels, as well as in 
the classroom. The initiative focused on four components, which are summarized below. 

Reviewing Policies and Programs 

With the Policy and Program Review Component, project leaders hoped to increase the “power 
and coherence” of state and local policies in support of the MSSI vision for science and 
mathematics education reform. A Policy and Program Review Group, based at Michigan State 
University, would review existing policies, programs, and regulations; identify strengths and 
areas of misalignment; develop recommendations for refining existing policies or passing new 
legislation to support reform; and annually assess progress toward policy alignment. The Policy 
Review Group would also examine the policy context in selected school districts, the impact of 
state policies on these districts, and local efforts to incorporate policies that reflected the MSSI 
vision. 

Creating Models of Effective Learning 

The Models of Effective Learning Component was to create “striking existence proofs” for the 
MS Si’s proposition that disadvantaged students can achieve at high levels in science and 
mathematics. Equity was a key consideration in the selection of participating “Focus Districts,” 
with competitive grants awarded to urban or extreme rural school districts with high numbers of 
underserved students. The MSSI’s premise was to launch reform in a small number of districts, 
refine strategies, and use these models to disseminate lessons learned, and achieve a broader 
impact. 

Redesigning Teacher Education 

The Teacher Education Redesign (TER) Component targeted both pre- and in-service education 
programs. A Teacher Education Framework Group (TEFG) would be responsible for developing 
a shared vision among universities, the Michigan Department of Education (MDE), and teacher 
associations; for creating a framework to guide the implementation of the vision; and for seeking 
the endorsement of high-level stakeholders for changes in teacher preparation programs. In 
translating the framework into practice, the TEFG would develop pre-service courses reflecting 
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the vision; provide opportunities for collaboration by scientists, mathematicians, and teacher 
educators; and conduct ongoing seminars to meet the needs of practicing teachers. 

Dissemination and Professional Development 

The Dissemination and Professional Development Component would serve as a major strategy 
for “propagating the vision.” Reform leaders expected to have much to share about instructional 
innovations, strategies for mobilizing parents and communities, professional development needs 
and technical assistance strategies, policy implications for schools and districts engaged in 
reform, and other relevant topics. The MSSI envisioned the dissemination of products through 
professional development activities with existing organizations and networks with ties to science 
and mathematics education. Through the dissemination component, MSSI leaders expected to 
foster commitment to reform, build “knowledgeable, well-prepared” coalitions of educators and 
community members, and facilitate inter-organizational connections. All of these efforts were 
expected to strengthen the infrastructure needed to support sustained systemic reform. 

Taken together, the four MSSI components sought to clarify vision; align policies; build local, 
regional, and statewide capacity; and spread the message. To those involved with the MSSI, 
these were the essential pieces for achieving system-wide change. Policies provided leverage. 
Focus Districts provided the opportunity to apply policies in real and practical ways. 
Infrastructure and capacity building would enable the scale up and institutionalization of reform 
activities. 

Woven throughout the MSSI design is the element of learning about the system, while also 
seeking to influence its direction. From the beginning, MSSI staff was “committed to the 
learning piece.” The “researchy” aspects in part reflected the interests and involvement of 
university faculty who pushed for the systematic study of the system, and the “discovery” of how 
policies and reform strategies might play out, particularly in the Focus Districts. By fostering 
changes in policies and practices in selected districts, the MSSI hoped to support and learn from 
local systemic efforts to address the needs of underserved students. Said this MSSI staff person: 

We were trying to learn about systemic reform. We designed something where we were 
both learning and developing understanding. . .We wanted to look at urban and rural 
districts and find out why kids aren ’t learning, and to work with a cadre of schools to 
bring about reform, and learn from them — to use the data to help with policy. We were 
working with them and learning from them — like a little learning experiment, a research 
study. 

Similarly, the policy review component focused on understanding the system. Through the study 
and interpretation of state, district, and local policies, the MSSI would create “informative” 
documents that would describe policy needs in light of the MSSI vision; identify the 
“characteristics of successful and counterproductive” policies; analyze the impact of state 
policies on schools and districts; and examine the experiences of Focus Districts in implementing 
reform. All of this information would be used to enlighten and engage stakeholders, and build 
support for the MSSI vision. MSSI leaders would in turn use the findings to leverage the system 
in appropriate ways. 
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A second common thread in the MSSI design was the emphasis on building system capacity by 
working with “middle level” individuals and institutions to extend the impact of reform 
activities. Again, reform leaders looked to the existing system, pinpointed needs and 
opportunities, and sculpted their role. With the professional development component, the 
choices were in part logistical. In the eyes of MSSI planners, the sheer size of the state and the 
number of teachers of mathematics and science precluded direct services. More importantly, the 
infrastructure was in place for delivering teacher professional development; some of the 
Mathematics and Science Centers were operating quite effectively, and there was evidence of the 
state’s plans to increase the number of Centers to extend their reach statewide. 

Finally, the Phase I proposal makes abundantly clear the MSSI’s intention to integrate equity 
throughout the design. Test scores at the time the MSSI was funded indicated a gap between 
affluent and poor districts, and there was a “fair amount of pressure” — both from the state and 
from NSF — to focus on the “weaker” districts, with the intended goal of creating “models of 
effective learning.” Recognizing that teachers in the most disadvantaged districts are the least 
well-prepared, the MSSI planned to improve professional development offered through the 
Mathematics and Science Centers, and leam more about the needs of teachers and students in 
Focus Districts in order to push for supportive policies. 

Over the course of the MSSI, the vision for reform never wavered. Equity remained central, as 
did the goals of bringing coherence to the system and strengthening the existing infrastructure to 
enable and sustain change. Refinements in implementation strategies came with time and 
experience as the realities of limited resources, coupled with immense needs, became more 
evident. If MSSI leaders began their mission a bit “starry eyed,” they soon learned that the task 
was more complex than they had imagined. Said one person: “Over the five years, we became 
more sophisticated at thinking more strategically. At first, we thought, ‘We have 10 million 
dollars and we have five years. That’s plenty of time and money to turn the system around. 

Then, [we were asking], ‘Where are the pressure points? Where are the levers? How can we take 
our meager resources and get the biggest bang for the buck?”’ 



The Realities of Implementation: 

Challenges, Trade-offs, Shifting Strategies 

Managing the Reform 

The MSSI was funded through the Michigan Department of Education’s Bureau of Instructional 
Services, and supervised by the Chief Deputy Superintendent of Public Instruction — the 
designated Principal Investigator (PI). The PI and MSSI Project Coordinator (who was then the 
MDE Mathematics and Science Coordinator) had worked closely with each other in the past, 
were well acquainted with the state policies and bureaucracy, and had extensive knowledge of 
the needs, resources, and opportunities within the system. One person described the leadership 
at the Department of Education at that time as “strong and stable, with the vision and energy to 
pursue systemic reform.” 

In addition to the PI and Project Coordinator, Co-PIs representing the university, business, and 
school communities were part of the early Management Team, as were faculty from Michigan 
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State University who were charged with Teacher Education Redesign and the Policy and 
Program Review Components. Members of the evaluation team from Western Michigan 
University were also actively engaged on the Management Team. Because the MSSI component 
leaders were geographically dispersed. Management Team meetings provided the primary forum 
for decision-making. It was the time and place where “everything got hashed out.” The 
Management Team met on a monthly basis (with core team members meeting more frequently), 
and involved a variety of formats, including “free-ranging” discussions, working sessions, 
presentations, and debates about strategies and direction, all driven by data collected and 
reported by the project evaluators. 

The Management Team provided a primary vehicle for linking a wide array of stakeholders, and 
in the words of the Abt monitor, this group was “rich both in the types of expertise involved and 
in the actual people who participated.” But the loss of three co-PIs early in the reform due to 
changes in employment prompted the MSSI to rethink the composition of the Management 
Team. There was a growing awareness of the need to include a wider range of players in key 
decision-making roles. New members of the Management Team included the MDE science and 
mathematics staff and the coordinator for Eisenhower programs, as well as representatives from 
other groups actively engaged in science and mathematics education — the regional Centers, the 
professional associations, the Detroit Urban Systemic Initiative — and parent and business 
representatives. 

In addition to the Management Team, the MSSI designated a Steering Committee comprised of 
representatives from the Governor’s office, State and local boards of education, universities, 
schools, businesses, parents, labor, and the media. As a vehicle for engaging an array of 
stakeholders, the Committee was to review and approve policy documents, make 
recommendations, and provide guidance on project components. In reality, the Steering 
Committee fell short of these expectations, with periods of active involvement alternating with 
inactivity and indecision about mission, roles, and expectations. Said one person, “Systemic 
reform is so complex. People didn’t even know the term when we first started. And we had to 
figure out how to inform them to get input and get them engaged. We struggled with that.” 

In a 1994 retreat, the Steering Committee redefined itself as “advisers, advocates, and 
ambassadors” for systemic reform. As such, they were to spread the message, expand the 
audience, and garner support. A set of bylaws developed at this time set the stage for the group 
to continue in these roles, establishing a prototype for a non-profit organization to sustain their 
work beyond the SSI grant. Despite these bolstering efforts, the Committee voted to disband, in 
part because the Management Team had broadened its membership to include other major 
players in the science and mathematics arena, and in doing so, had increased its own capacity to 
guide reform efforts. 

Making Connections 

From the outset, the MSSI was “about making connections.” Reform leaders envisioned a role 
within and across project components as one of facilitator and convener — bridging policy areas, 
linking policy with practice, and connecting key players in reform. From this vantage point, the 
MSSI viewed itself as a critical “enabler” of reform, providing a structure and vehicle to promote 
connections aimed at strengthening the mathematics and science education system. In the eyes 




of MSSI staff, their role was to “build relationships, catalyze conversations, and foster thinking 
outside of people’s own little domain.” Said the Project Coordinator: “We were trying to bring 
people together to collaborate, to work collectively in the same direction. The MSSI was 
designed for this.” 

Housed in the MDE, the MSSI was well positioned to link others in the Department who 
provided direct and indirect support to mathematics and science programs — curriculum directors, 
as well as staff connected with Title I, Title II, School to Work, and Goals 2000. At the same 
time, however, the MSSI had difficulty establishing an identity for itself separate from the MDE. 
Entrenched attitudes also deterred efforts to forge connections within the Department. In part, 
MSSI leaders were “learning as we were going.” For example, neglecting to involve MDE 
curricular staff on the Management Team from the outset proved to be a valuable lesson. The 
oversight resulted in an initial lack of buy-in and participation, even after the MSSI sought to 
involve them. Said one person, “Some looked at the SSI and projected that it would be done in 
five years, so there was no need to get involved. [They were saying] ‘Why should we change?’ 
There could have been better ways to engage them.” 

The MSSI experienced similar challenges with the state-funded regional Mathematics and 
Science Centers. The network of 25 Centers and 8 Satellites serve Michigan’s districts 
statewide, providing leadership and professional development, support in curriculum 
development and community involvement, student services, and resources in science and 
mathematics. As state-funded entities, the Centers had embraced their regional roles, but had yet 
to achieve a true network driven by a common vision. Early in the MSSI, the Centers viewed the 
initiative as another short-lived project in the reform landscape. Turf issues and resistance to a 
systemic approach created tension. MSSI leaders persisted, however, recognizing the 
significance of the Centers to their effort. Said one of the evaluators: 

Very early on, the SSI said these Mathematics and Science Centers are a critical piece of 
this thing. If we ’re ever going to make anything happen, we 've got to maintain this piece 
of infrastructure. The SSI went to bat frequently for the Centers to get funding for them, 
but the Centers saw the SSI as a competitor. 

To engage the Centers in reform, the MSSI recruited regional directors to participate on the 
Management Team. Through a series of one- and two-day statewide institutes and summits, the 
MSSI also worked to promote a common vision, for example, by engaging staff from the Centers 
around professional development issues. The goal was to build a statewide network that would 
be “less subject to local idiosyncrasies and resources” — one that would “put people on the same 
page.” 

The MSSI also viewed the state’s professional associations as key players in reform, and sought 
to forge connections between MSTA and MCTM. At the time the MSSI was funded, there was 
“virtually no communication” between the science and mathematics professional associations in 
the state. Remembers one MSSI staff person: 

There was no understanding, no leveraging of resources, no joint efforts. There was no 
hostility — it was just a separate but equal kind of mentality. ‘They do our thing, we do 
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ours. ’ We came in believing that as one small piece of our effort, we could act to broker 
the efforts of these two groups to work together and to think strategically and to put their 
interests together in a way that would expand the outcomes. 

Convincing people of the value of collaboration took “more effort than anyone imagined.” 
Leaders in professional organizations were strong-minded, and slow to warm to the notions of 
strategic planning to link mathematics and science education. The MSSI used several strategies 
for engaging these players, however, with large pay-offs. They recruited MCTM and MSTA 
leaders as members of the Management Team, involving them in critical decision making roles 
and reflective discussions around vision and planning. The MSSI also provided deliberate 
opportunities for MSTA and MCTM representatives to collaborate — for example, co-sponsoring 
events for professional development providers — to give them the “opportunity to think about 
how they were intertwined.” In large part, these efforts helped MSTA and MCTM overcome 
recurring turf issues. Prior to the MSSI, the “default” was “Let’s go at it alone, why should we 
help anyone else?” While collaboration is not necessarily the current default, it comes “earlier in 
the conversation” as players seek to “maximize their gain.” 

Revamping Teacher Education 

The higher education system in Michigan includes 15 publicly funded universities with teacher 
education programs. At the time the MSSI was funded, the state was generating high numbers of 
credentialed teachers, but in the eyes of MSSI staff, the programs were “way behind the curve.” 
The Teacher Education Redesign (TER) component sought to develop a shared vision for the 
preparation of mathematics and science teachers, and create guidelines to help universities 
implement this vision. Representatives from public universities, private colleges, and 
community colleges convened for these purposes at three MSSI-sponsored conferences. 

From the MSSI meetings emerged a “consensus” document entitled “Guidelines for Science and 
Mathematics Teacher Preparation in Michigan.” The document incorporates the MSSI vision for 
high quality mathematics and science education for all students, and outlines major goals for 
transforming the teacher education system. Universities that chose to implement these strategies 
formed interdisciplinary teams to translate the recommendations into practice. Said one faculty 
member about the nature of this work: 

Our focus was the quality of experience teachers were getting, and less so on 
accreditation and course requirements. It goes back to the issue of capacity building and 
infrastructure. The fundamental problem wasn ’t irrational teacher certification 
guidelines. The reed barriers were that people lacked capacity to provide quedity 
education programs for our students. That’s where we made progress and that’s where 
we needed to make progress. 

Like other MSSI components, the TER focused on promoting interaction. Within universities, 
reform teams included faculty representing the content, pedagogy, and practicum elements of the 
pre-service experience. At biannual MSSI conferences, job-alike faculty representing each of 
these areas met to refine vision and strategies. These opportunities were highly valued, 
particularly for faculty from regional institutions who rarely had opportunities for meeting with 
colleagues to discuss shared problems and issues. Community colleges that pushed for 
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involvement in the MS SI implemented changes in introductory courses articulated with those in 
four- year institutions. 

Over the course of the MSSI, the Teacher Education Redesign component evolved into a formal 
network known as the Michigan Teacher Preparation Collaborative. The Collaborative 
symbolized the MSSI’s success at establishing new relationships among higher education 
institutions in the interest of pre- service reform. Composed of public universities, independent 
colleges, community colleges, and Local Alliances of universities and urban school districts, the 
Collaborative sought outside funding to extend pre-service education reform efforts and sustain 
local networks. 

Still, there were challenges. With limited time and resources from the MSSI and from outside 
sources, the majority of the teams had difficulty building on their work. Individual faculty 
involved in the TER noted making changes in their own courses and teaching habits, but the 
Collaborative had difficulty securing outside funding to expand on these efforts, and no longer 
meets formally. And while the MSSI encouraged the work of the Local Alliances, few of these 
groups remain active today. In the end, three of the universities involved in the TER sought to 
institutionalize changes — two with FIPSE grants used to restructure science courses for 
elementary school teachers. The third launched a comprehensive reform of pre-service 
education in science, mathematics, engineering, and technology. 

Shifting the Professional Development Paradigm 

The original MSSI plan combined professional development and dissemination under one 
component, where professional development activities would be used to disseminate what was 
learned through various reform activities. Reform leaders had intentionally chosen to avoid the 
delivery of direct services — primarily because Michigan already had much to offer in 
professional development, and because NSF discouraged project leaders from focusing on 
teacher enhancement. Over time, however, it became increasingly clear that the nature of 
existing professional development activities conflicted with the MSSI vision, prompting staff to 
rethink their strategy and take a more active role with professional development providers in the 
state. After the first year of implementation, the MSSI created two separate components: 
professional development and communications. The latter, aimed at developing awareness and 
understanding of the MSSI vision, occurred through a public relations campaign; through work 
with Focus Districts on engaging the community; and through publications that included 
quarterly MSSI newsletters and “Quick Facts,” a source of practical information for Focus 
Districts, Center directors, and others. 

Strategies for strengthening the state’s professional development infrastructure emerged from 
Management Team discussions over the first year of the MSSI. Like the rest of the system, 
professional development activities in Michigan were fragmented, piecemeal, and of varying 
quality. Multiple providers included Mathematics and Science Centers, Intermediate School 
Districts, universities, and district curriculum departments. In mapping the professional 
development system, MSSI staff found that it “looked like a pile of spaghetti — a million things 
with no coherence.” 
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There were “fits and starts” as the MSSI tried to define a reasonable role for itself. It was one 
thing to provide professional development and quite another to increase the capacity of the 
system. Settling on a strategy of working with professional development providers, including 
MSTA, MCTM, and the existing Mathematics and Science Centers, reform leaders sought not 
only to build a stronger professional development system, but also to change the nature and 
quality of what was offered. Remembers one of the professional development co-directors: 

Most of what we found was embedded in the traditional model of the one-shot workshop, 
expert lecture, the university model. The need was clear to work with these intermediate 
organizations and individuals who rubbed up against school districts, who had 
professional development roles and responsibilities and to build their capacity and 
develop strong communication across various players. 

Initially, the MSSI disseminated their new vision for professional development through the 
project’s statewide newsletter. In 1994, project leaders co-sponsored a two-day conference for 
participants to learn about newly emerging best practices (e.g., ongoing, job-embedded 
professional development), and to initiate the development of statewide professional 
development guidelines. Subsequent meetings convened by the MSSI for state leaders in 
mathematics, science, and professional development identified “action steps” needed to overhaul 
the professional development system. MSTA, MCTM, and MDE curriculum development staff 
and Eisenhower coordinators were all involved in these efforts, attending quarterly meetings 
with MSSI co-directors to plan and refine strategies for building learning communities through 
the Mathematics and Science Centers charged with regional staff development efforts. By 
design, MSSI meetings also served as opportunities for building communities of learners among 
professional development providers themselves, who were becoming more unified in their vision 
and strategies for teacher development. 

An outgrowth of the 1994 meeting was the creation of Regional Collaboratives. Comprised of 
the Mathematics and Science Centers and professional associations, the Collaboratives were to 
provide coordinated support and help build district capacity to implement change. MSSI staff 
also made efforts to disseminate their vision for professional development throughout the system 
by working with MDE curriculum and professional development staff and Eisenhower 
coordinators, by participating in state level policy discussions, and through newsletters and a 
Dialogue Web. 

Through all of these activities, the MSSI contributed to a “rising awareness in the field.” Said 
one person, “It’s a different environment now. There’s more sophistication in some districts 
about what they’re looking for and what they want in professional development. The 
Mathematics and Science Centers are building the capacity of teachers to examine student work, 
and teachers are coming together on a monthly basis. That would have been unheard of in 
1992.” 

A major challenge, however, lay in communicating the vision for professional development to 
the districts and schools. As professional development providers began to transition from the 
“one-shot dose” to a more complex, job-embedded model, they encountered some resistance. 
Said a co-director: 



12 




Districts were still expecting the traditional model, or a “How to Succeed at MEAP 
workshop. ” So when providers said, “Let’s work with teachers over the year, one 
afternoon a month, examine student work, ” things like that, the districts would say, “ Are 
you crazy? Can ’tyou just come for one and a half hours after school one day?” So the 
field is pretty unsophisticated. They know one way and that’s what they understand. 

They ’re confounded by a new vision for professional development. 

An added challenge was the level at which the MSSI focused its efforts in the professional 
development arena. The approach was decidedly systemic and sought to broaden the reach of 
reform — by working with existing infrastructure and increasing the capacity of individuals and 
institutions charged with providing professional development. The approach of working with the 
“middle level,” however, posed problems for measuring direct impact on teachers. Further, 
teacher participation in professional development activities in the state was for the most part 
voluntary, thereby challenging the MSSI to address critical needs among teachers of science and 
mathematics through their work with professional development providers. 

Ironically, by the end of the MSSI, reform leaders had come to believe in the greater value of 
focusing professional development efforts at the school level, as opposed to the system level. 

Said the Abt monitor in a “reflective memo” to NSF in 1997 — at the end of the MSSI: 
“Intervention in the system of professional development began as a strategy of strengthening 
traditional providers such as the Regional Centers. However, after three years, the developers 
became convinced that the success of professional development in promoting systemic reform 
actually depended on using the school as the unit of development and fostering learning 
communities.” 

Beyond Vision and Policy: Supporting Focus Districts 

The MSSI spend the first two years of the initiative building consensus around the vision, with 
the intended effect of unifying mathematics and science reform across the state. Discussions 
occurred in a variety of formats and involved high-level stakeholders — MDE staff, professional 
organization leaders, Mathematics and Science Center directors, university faculty and 
administrators, and business representatives. At the same time, the policy and program review 
group mapped state policies and examined the ways in which they supported or hindered reform. 
Phase I of the policy component focused on state level policies; the first policy report was widely 
disseminated to legislators, professional associations, and educators, with briefings and MSSI- 
sponsored meetings on policy alignment, barriers, and systemic change. 

In the eyes of MSSI staff, the policy component was critical for assessing the existing system 
and reinforcing the vision. However, there was also a growing recognition of the need to 
increase the relevance of the documents generated by the policy group, with a shift to a more 
practical, applied approach. Said one of the evaluators: 

The policy piece started out as a research effort. But there was pressure on the 
university people involved in Management Team discussions. They were saying if we ’re 
going to do these studies, we’ve got to turn them into something that’s going to be useful 
to people. So the emphasis changed to be more practical. They made a serious attempt 
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to translate what they had learned into forms that could be digested by people on the 
front line. But when they started, they were thinking much more esoteric. 

With a move toward “the applied,” subsequent policy reports focused on districts and schools. 

In Phase II of the policy component, for example, the MSSI examined the impact of federal and 
state policies in nine districts (three of which were Focus Districts), and encouraged Focus 
Districts to review their own policies and programs with respect to the MSSI vision. In Phase 
III, the policy group examined impact at the school and classroom levels. The resulting 
documents to emerge from these efforts revealed similar findings: more than state policies, 
district capacity to support reform was the critical element in improving mathematics and science 
education programs. 

The Focus Districts provided the MSSI with opportunities for studying and influencing reform at 
the local level. These “Models of Effective Learning” included 11“ high needs” Target 
Districts, which received up to $65,000 each year over a four year period; 13 Affiliate Districts, 
whose needs were less pressing, received up to $3,000 to support reform-based activities. All 
selected districts were required to develop vision statements, build local coalitions of 
stakeholders, and work with a small set of schools to improve curriculum, instruction, 
assessment, community support, school organization, and resource allocation. In addition to 
funding. Focus Districts received technical assistance through the MSSI to support their efforts. 

The Models of Effective Learning was the largest of the MSSI components, with almost half of 
the NSF resources directed to the targeted districts. Capacity building efforts with Focus 
Districts centered on developing local Science and Mathematics Coalitions. Comprised of 
representatives from schools, professional associations, business and industry, community-based 
organizations, institutions of higher education, and others, the Coalitions were to create strategic 
plans for addressing equity, systemic reform, and community engagement; modify policies and 
practices in support of science and mathematics; and raise matching funds from local sources. In 
the eyes of reform leaders, these Coalitions would provide the structure for supporting and 
sustaining local reform efforts. 

The MSSI held a total of nine conferences for targeted districts, providing cross-district 
networking opportunities and introducing teams from each district to a range of relevant topics, 
including the MSSI vision for curriculum, instruction, and professional development; strategic 
thinking, planning, and evaluation; and strategies for sustaining and scaling up reform activities 
beyond the NSF grant. District representatives were to return to their communities to implement 
what they had learned. 

In their initial planning, MSSI leaders believed that the Models of Effective Learning would 
provide “existence proofs” — showing the way for other districts to improve teaching and 
learning. But reform efforts in the Focus Districts varied considerably, depending on context and 
capacity. For example, in Detroit — a district already well embarked on mathematics and science 
education reform — MSSI funds were used to support the work of the Urban Systemic Planning 
team in the area of community outreach. Other less experienced Focus Districts had to start from 
scratch. In these cases, MSSI funds supported efforts to build awareness of a new vision, to 
develop curricular frameworks aligned with the state’s standards, and to provide professional 
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development. In some of these cases, however, the barriers and needs far outweighed the 
available resources. 

In hindsight, MSSI staff reflected that they had underestimated the level of effort needed for 
change to occur locally. The MSSI had “taken on the toughest districts in the state,” where 
planning and implementing systemic reform was “foreign territory.” Conference participants 
often returned to their districts to find little local support for implementing new practices. For 
example, school and district administrators often had a limited understanding of the kinds of 
support needed to implement change; without this support, teacher participation and commitment 
was weak. Further, building effective Coalitions took considerable effort — starting with getting 
the “right people” on the teams: 

You have to have a certain set of people on the team. In the districts where there was a 
good mix of teachers, principals, community members, you saw a lot more progress. But 
if you didn ’t have some policy makers on your team, you could go back and talk until you 
were blue and not get anywhere. By the second or so year into funding for the Focus 
Districts, the SSI formed a superintendents group. It was quite obvious that where the 
superintendent was involved, it made a difference. (MSSI Evaluator) 

When it came to bringing in the local school boards and the community at large, we 
could have worked more in those areas. If you want to have systemic reform, you really 
need to get the decision makers — the policymakers, the district boards — involved in the 
process and get their buy-in. (MSSI Steering Committee member) 

District readiness, the size and scope of MSSI grants, and the misalignment of MSSI work with 
local priorities and politics also influenced the extent of impact. Again, in part, the challenge lay 
in getting the vision and policy “down to the classroom.” For example, districts struggled with 
how to translate standards into curriculum and were typically unprepared to do so. Said these 
two persons associated with the MSSI: 

I’m not sure people realized how important it was to have an articulated curriculum in 
place. You could do all this instruction, have programs and policies, and it wouldn 7 
make any difference. A K-12 curriculum is pretty critical. Without that, all the rest of it 
is pretty irrelevant. When 1 look back, nobody was saying that this was critically 
important, although some districts embarked on that. (MSSI Evaluator) 

If people don 7 know what the standards are, they ’re not going to implement them. There 
has to be capacity to understand what it looks like in the classroom, how to do it, and 
how to implement it. That was a key finding. Teachers struggled to change their 
practice. They were implementing what they thought were standards, but they didn 7 
understand them well enough to do it well. (MSSI Project Coordinator) 

While conferences were intended to bolster these kinds of skills, it was the follow-up support 
that proved to be the crucial missing piece. Focus Districts needed “a lot more nudging and 
support” in between meetings, but the MSSI had never envisioned a strong technical assistance 
role for itself. Initially, reform leaders sought to recruit high performing districts to fill this 
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function, acting as “mentors” to high needs districts. But there were no incentives for mentoring 
districts, and the responsibility for providing support fell to the shoulders of MSSI staff. Said the 
Project Coordinator: “We found providing technical assistance very tricky. It was hard to define 
what districts really needed, and hard to pull in the right people. We had meetings at the districts 
and invited mathematics and science people and others with support roles, but there was not 
much incentive for them to participate. Sometimes they came, sometimes they didn’t.” During 
the last years of the MSSI, as needs became more apparent, staff stepped up technical assistance 
efforts, providing grant money to selected Focus Districts to hire consultants to help them 
develop and implement their strategic plans. 

The Highs and Lows of Evaluation 

Based at Western Michigan University, the evaluation team participated as members of the MSSI 
Management Team from the outset. The evaluators collected extensive data, provided 
information “from the field,” facilitated strategic planning within and across the MSSI 
components, and contributed heavily to project decision-making. Working with component 
coordinators, Focus Districts, the Michigan Department of Education, and the Science and 
Mathematics Centers, evaluators helped determine evaluation needs and improve data collection 
strategies. They attended MSSI meetings and conferences, and spent considerable time in the 
Focus Districts, looking at processes, plans, activities, and decision-making, and collecting data 
on rates of participation in science and mathematics courses, student and teacher demographics, 
program characteristics, student and teacher attitudes, student achievement, teacher 
accomplishments, and community involvement. 

Over time, the evaluation gained saliency. Evaluators helped the Management Team assess the 
effectiveness of reform strategies and consider alternative strategies. According to the MSSI 
Project Coordinator, the evaluation was a “major component that helped contribute to what we 
did and where we needed to make changes.” Fine-tuning was recurrent, depending on feedback 
from the evaluation team. Said one member of the Steering Committee: “The evaluators 
provided us opportunities to look at where we were, ask ourselves if we were moving toward 
reaching our goals. There was always an effort to look at whether or not we were on the right 
track, asking what can we do differently. That was ever present.” The evaluators were 
themselves duly impressed with how receptive the MSSI was to the information provided, and 
the Management Team’s systematic use of data in decision-making. 

NSF concerns and shifting priorities also played a role in the evaluation. While these “didn’t 
derail the basic approach,” they did influence MSSI decisions about how to demonstrate 
progress. Said one person, “NSF needs were always there and they were always changing. We 
were always jumping to their demands, whatever they were.” An evolving set of NSF program 
officers assigned to the MSSI complicated the task of communicating project vision and 
strategies. Wrote the Abt monitor of the challenges associated with rotating NSF staff: “In the 
last two years, there were three different program officers, [and] the talk at MSSI was always 
about educating the new Program Officer and bringing him or her up to speed on the project. . .It 
is not clear that any of the successors ever did become intimate enough with MSSI to advocate 
for it at the federal level. . .MSSI is a complicated, multi-faceted initiative and I’m not sure that 
any of the later program officers ever had time to become completely familiar with it.” 
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The “attribution” issue — NSF’s emphasis on the impact of systemic reform activities on student 
achievement — was a particularly thorny one for the MSSI. Reform leaders deliberately chose to 
work on capacity and infrastructure at the intermediate levels, making it more difficult to 
determine the MSSI’s influence at the classroom level. In the words of the Abt monitor, “The 
numbers game didn’t work in Michigan.” In addition, there were numerous activities in the state 
focused on improving science and mathematics education. In the eyes of MSSI staff, these 
conditions also made it difficult to attribute change directly to their initiative. Further, assuming 
credit for improvements in this context was a delicate issue, sometimes rankling others engaged 
in reform efforts and rekindling turf issues. Said the MSSI PI: “We just had too many things 
going on in the state to give all the credit to [the MSSI].” 



What’s Changed? 
The Impact of the MSSI 



Policy and Infrastructure 

The MSSI exerted its influence across the education system. MSSI “vision” and policy papers 
promoted a clearer mission and direction for reform. Further, through the participation of 
Management Team members on state level committees, the MSSI influenced the development of 
the Michigan Curriculum Standards and Frameworks, and widely distributed these documents 
through their work with state and local stakeholders. Management Team members were also 
involved as advisors and reviewers for the science and mathematics MEAP test revisions, 
promoting alignment with state Standards and Frameworks. Finally, MSSI leaders were well- 
represented on the task force to develop Michigan’s Professional Development Standards, 
adopted by the State Board of Education in 1995; these Standards promote capacity building, 
equity, and “life-long learning” opportunities for teachers. 

The MSSI influenced the system in other important ways as well. Funded through the Michigan 
Department of Education, the project was well placed to promote its vision for reform, and 
helped shape the reorganization of the MDE. The restructured Department offers greater 
potential for strengthening professional development and school-based activities, calling for 
teamwork and “continuous learning organizations.” Further, the MSSI helped promote a more 
cohesive approach to supporting schools; MDE Field Services Units now provide technical 
assistance to local and intermediate school districts, helping them to coordinate the efforts of 
federal, state, regional, and local initiatives. Still, as one MSSI staff person said of these 
changes: “Some of the influence was subtle and is an example of the type of change that is hard 
to attribute to the MSSI.” 

Beyond question, the MSSI was instrumental in strengthening the work of the Mathematics and 
Science Centers, and reform leaders used the Centers to disseminate their vision to Michigan’s 
45,000 teachers of mathematics and science. Today, the 25 Centers and 8 Satellites are 
considerably stronger in vision and capacity than they were prior to the MSSI. Said one MSSI 
component coordinator: “We didn’t get there by the end of the SSI. It just took a lot longer than 
we thought. But we made progress and [promoted] a lot of conversations that would not have 
happened without us.” More “sophisticated” in thinking and “tuned into best practices,” the 
Centers have shifted their paradigm, focusing on “facilitating teacher conversation — helping 
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teachers take the learning back to their school so they can continue the conversation.” Project 
leaders also wielded their influence in support of the Centers, pushing for expanded state funding 
at the time of reauthorization to help sustain the structures. In addition, the MSSI was able to 
leverage funds for the development of curriculum frameworks, and for professional development 
through $9.5 million from the Eisenhower program. 

Similarly, the MSSI significantly strengthened leadership and capacity among Michigan’s other 
major professional development providers. Within the mathematics and science communities, 
MSSI representatives described an expanded vision: projects support a systemic approach and 
leaders seek to bridge K-12 and higher education. Said one person, “I don’t think you can say 
that changes in professional development in Michigan are all attributable to the SSI, but they 
would not have happened at the level they did without the MSSI. Nobody else was there to pull 
the players together.” 

Such was the case with pre-service education reform as well. The MSSI was instrumental in 
convening university teams, conveying a new vision for preparing science and mathematics 
teachers, and developing guidelines for pre-service reform. TER teams created a “Blueprint” for 
teacher education reform, built awareness among university faculty and administrators about the 
need for change, and forged stronger links both within and across universities and community 
colleges. Three universities implemented major changes in their pre-service education programs 
as a direct result of their involvement in the MSSI. 

Districts and Schools 

The MSSI engaged districts with the greatest needs, and provided them with access to resources 
and technical assistance. Regional networks, conferences, workshops, and district-university 
collaborative efforts all provided capacity building opportunities for teachers, administrators, and 
others. Evaluation data revealed that two-thirds of the Focus Districts made moderate to 
substantial progress in strengthening curriculum and instruction, administrative support, 
professional development, and parent-community involvement. For example, the majority of the 
districts added enrichment programs in mathematics and science, initiated structural and 
organizational changes to allow shared planning times for teachers, and supported regularly 
scheduled professional development. As a result of their participation, teachers reported using 
new instructional practices; schools sought out community resources, aligned curriculum with 
standards, and created new science- and mathematics-related activities for students and the 
community to generate interest and support. 

Through the MSSI, Focus Districts were able to leverage other funds to support their work; for 
example, in 1994-95 alone, these districts leveraged over $7 million — through Eisenhower and 
other federal funds, partnerships, foundations, and grants. A grant from the American 
Association for the Advancement of Science allowed the MSSI to expand its equity work 
through conferences sponsored by the Mathematics and Science Centers and professional 
associations, and through the development of a “toolkit” to help classroom teachers adopt more 
equitable classroom practices. 

The MSSI also reported small gains in MEAP scores in Focus Districts that received the highest 
levels of support, and a slight reduction in the performance gap between white and minority 
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students. Nine of the 1 1 Target Districts reported increases in the number of students who 
passed the 11 th grade science and mathematics High School Proficiency Test, and six of these 11 
exceeded the state average. All of these data suggest that the MSSI may have had some impact 
on improving student achievement in Michigan’s most disadvantaged districts. 

Despite these successes, MSSI efforts to create “models of effective learning” had mixed results. 
Some districts “took full advantage” of what the MSSI had to offer, while others “never had the 
creative energy” to implement new policies and practices. Districts with strong local Coalitions, 
leadership, and commitment from high level stakeholders fared better than others. But the 
implementation of high quality, standards-based curricula remained a huge challenge, requiring 
not just teacher capacity, but also administrative, organizational, and logistical support. The 
districts were “tyrannized by the MEAP,” and many lacked the capacity, will, and readiness for 
major changes envisioned by the MSSI. And while teachers demonstrated changes in 
instruction, they “had a long way to go” before fully matching classroom practices with the 
MSSI’s standards-based vision. 

By the time Focus Districts began to implement their strategic plans, three years remained on the 
MSSI grant; according to one person closely involved with these districts, there was “simply not 
enough time to see real change.” In hindsight, expectations for major changes at the school and 
classroom levels were unrealistic; hopes for significantly improving test scores and reducing 
performance gaps had been “a pie in the sky, to say the least.” In the state’s most disadvantaged 
districts, without higher levels of direct intervention, professional development, and technical 
assistance, those kinds of changes just “weren’t likely to happen.” 

The MSSI raised awareness in the districts, however, and helped build networks of stakeholders 
locally. But even these changes were difficult to sustain. To the end, the MSSI struggled with 
identifying common needs, finding appropriate ways to provide support, and building self- 
sustaining networks both within and across Focus Districts. MSSI representatives revealed that, 
while there may be “remnants of the things they put in place,” the legacy at the district level was 
minimal. Turnover among local leaders, and inadequate time, resources, and support all 
contributed to the backslide. Said one person, “We learned a tremendous amount about what 
you have to do in order to move a school or a district, but I’m not sure we moved the ones we 
worked with.” 

The Challenges of Sustaining Change 

The MSSI carved out a role for itself as collaborator and catalyst, forging connections across 
groups working toward the same ends. Said one person: “What makes a difference is when you 
bring people together around a real problem, help them organize their thoughts into an action 
plan, and follow up in implementation. Those are good strategies. They allow people to design 
solutions collectively.” The MSSI firmly believed in the need for a structure to sustain this 
function, and created a non-profit organization — the Michigan Science and Mathematics 
Alliance (MiSMA) — to assume this role beyond the SSI grant. 

With seed money from the MDE, and with representation from the Mathematics and Science 
Centers, MCTM, MSTA and others associated with the Management Team, MiSMA held 
promise for sustaining the structure and function of the MSSI. From its earliest days, however, 
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MiSMA had difficulty defining its niche. Turf issues again arose, with other state players 
questioning the need for “another layer.” The organization continued to exist for a number of 
years after funding for the MSSI ended; however, MiSMA did not fulfill expectations for 
sustaining the work of the MSSI, and recently disbanded altogether due to a lack of funding to 
support paid staff. 

The need for an entity to carry out the “convening” role was highlighted at the end of the MSSI, 
when the policy group published its final report on “lessons learned” about implementing 
instructional change in science and mathematics. While the report was widely disseminated to 
Focus Districts and others, there was no structure in place for reflective dialogue or follow-up. 
Commented these two persons associated with the MSSI: 

Systemic change takes a long time. Five years gets you started, then you have that long- 
term implementation process, with people wrapping their heads around it, and getting 
people connected, and having someone there to remind them, to organize them, to bring 
them together to talk and make plans, and reconvene them. No one ’s asking them to do 
that now. (MSSI Project Coordinator) 

There 's no question that work with leadership resulted in an impact on the 
organizations — MCTM, MSTA, the Mathematics and Science Centers — those 
organizations ’ capacity was increased. The downside was that we didn 7 influence the 
professional development system in Michigan or didn 7 bring much coherence to it. The 
other thing that's missing today — the one thing the SSI did do and nobody ’s doing it 
now — it was a way to convene people. If you don 7 convene people, you can 7 keep a 
common vision going. Now it only happens within an organization, and not at a higher 
level than that. The MDE has not taken that role. (MSSI Evaluator) 

Similarly, the Abt monitor described both the successes of the MSSI, as well as the challenges 
faced by project leaders in their efforts to sustain their work: 

MSSI was able to weave together networks and systems in a more effective way, support 
good practice in multiple arenas, unravel complexities of policy and practice, and build 
local capacity... Having put all these components into motion and “unfrozen” the 
traditional systems and their relationships to one another, what MSSI could not do was to 
pull it all together in the time that was available to them and make the outcomes credible 
to outsiders. At the end of five years, MSSI had built, created, analyzed, strengthened, 
and reinforced the critical systems, but had not yet seen the work come to fruition. 

(MSSI Abt Monitor) 

Today, aligned state Standards, Frameworks, and assessments in Michigan convey a common 
vision for science and mathematics education, and the Mathematics and Science Centers, MSTA, 
and MCTM continue to promote this vision through their work with teachers. For example, 
MSTA’s work through the NSTA-funded “Building a Presence in Science” is considered a 
“direct outgrowth of the MSSI,” and seeks to develop a cadre of building liaisons to facilitate 
and support ongoing professional development. Still, to some degree, fragmentation has 
returned to the system. Reflected one person, “We’ve regressed. There’s a lot going on with not 
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much coordination of effort. The Mathematics and Science Centers go about their thing and 
higher education does their thing and sometimes there’s overlap and sometimes there isn’t.” 



Factors that Shaped the Michigan SSI Story 

MSSI strategies for reform addressed major components system-wide, including policy, pre- 
service reform, professional development, and district-level “models for effective learning.” A 
number of factors contributed to both the successes and challenges the MSSI encountered in 
implementing their design. 

> The MSSI created a viable role for itself within the system, but had difficulty sustaining 
this identity amid existing educational reforms and infrastructure. 

The MSSI took stock of the system, identified needs and gaps, and concentrated resources on 
bolstering and aligning reform efforts that were already underway. Providing a unifying vision 
for science and mathematics reform, bringing equity to the forefront, and engaging relevant 
stakeholders all contributed to forward progress in Michigan’s broader reforms. Centrally 
located in the Michigan Department of Education, with high-level leadership and support in the 
Department, the MSSI was well-positioned to promote their vision for system-wide reform. 

Still, identity problems and turf issues came with the territory. While the MSSI sought to 
establish itself as the “umbrella,” it had to overcome perceptions that the initiative was another 
bureaucratic layer charged with mandating yet another short-lived reform. 

> Reform leaders sought to learn about systemic reform, but had difficulty applying these 
lessons in their work. 

From the start, MSSI staff was vigilant about collecting and reporting information that could 
make the system more amenable to reform. Project leaders used this information to educate 
stakeholders, clarify vision, build capacity, promote dialogue and reflection, gain consensus, and 
refine strategies. All of this information was of value for learning about the system, and for 
communicating what was required for implementing systemic reform. At the same time, the 
MSSI encountered obstacles in applying some of these lessons and in translating policy to 
practice — particularly at the district and school levels where more direct and sustained 
interventions were needed. Aligned policies and better-informed stakeholders did not 
necessarily guarantee changes in the classroom. Further, information was not always available 
in a timely way: “lessons learned” from the Focus Districts were not ready for dissemination 
until the final year of the MSSI, and by then, there was little time or opportunity for reform 
leaders and others to apply what had been learned. Finally, the MSSI’s system- level of attack 
posed challenges for measuring impact and attributing improvements to the initiative. 

> Reform leaders sought to make connections and bring cohesion to a fragmented system, 
but lack of sustained commitment by those within the system hindered these efforts. 

The MSSI saw coordination and collaboration as fundamental to lasting change. MSSI staff 
sought to articulate a shared vision and build connections — convening stakeholders, building 
teams and networks, fostering collaboration, and developing action plans based on shared 
beliefs. Through these efforts, the MSSI greatly increased dialogue and awareness in the policy 
arena, in the university community, with professional development providers, and among district 
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and school personnel. Still, collaboration is time-consuming, labor-intensive work, requiring a 
balance between discussion and action; some wondered whether the MSSI “over analyzed” about 
the types of structures to put in place to ensure that people engaged in conversation. Further, the 
success of these efforts is contingent on the participation of the “right people,” and their 
commitment to reform — a difficult lesson learned by the MSSI at the local level. Finally, 
networks must be willing to take on roles and functions that serve not just their own needs, but 
also the needs of the system. While the MSSI contributed to increased collaboration within 
universities and among professional development providers, coalitions that spanned system 
components — the MDE, state level organizations, universities, and school systems — were more 
tenuous. The MSSI’s difficulty in establishing an entity to help sustain the networks it had 
created diminished the likelihood that collaboration would continue over time. 

> Reform leaders focused on building capacity and infrastructure, but were unable to 
sustain these efforts beyond the grant. 

Building the capacity of individuals and organizations was a defining feature of the MSSI. With 
a stronger infrastructure, the system could better support, scale up, and sustain changes. Pushing 
for aligned policies and assessments provided a foundation for reform, and all of Michigan’s 
teachers and students were influenced through these policies. Similarly, by working with the 
university system and the professional development infrastructure, reform leaders expected to 
extend their reach. The strengths of the design lay in its system-level approach. At the same 
time, the approach (deliberately) focused little effort on direct intervention, and resulted in a 
critical challenge: translating system- level capacity building efforts into effective 
implementation at the classroom level. Working with high needs districts at various stages of 
readiness for reform, the MSSI struggled with how to reconcile local needs with resources. In 
the end, districts needed far more support than the MSSI was prepared to give. Further, as the 
MSSI looked to strengthen the professional development system by working with Science and 
Mathematics Centers, they underestimated barriers at the local level: entrenched attitudes, 
misaligned policies, and school cultures that inhibited the implementation of ongoing, site-based 
staff development. A Phase II proposal — not funded by NSF — planned to strengthen 
institutional relationships and build the capacity of people who could help reform “find its way 
into the classrooms.” But without continued funding to support these efforts, the likelihood of 
continued improvements at the district level was minimal. 
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Appendix J 



Case Report 

Nebraska Statewide Systemic Initiative 




Introduction 



In 1991, the University of Nebraska in Lincoln received $4.7 million from the National Science 
Foundation for a statewide systemic initiative (SSI). Funded with other Cohort I systemic 
initiatives, Nebraska initially focused on mathematics education reform. In 1993, the University 
received $5.3 million to extend reform efforts to science. The additional funds put the Nebraska 
Mathematics and Science Initiative (NMSI) on a six-year funding cycle ending in 1997. 

The NMSI sought to establish itself as the “catalyst and coordinator” of systemic change at the 
state, regional, and local levels. In a state with an abundance of small, isolated communities 
heavily vested in local control, the NMSI proposed a grassroots approach to improving 
mathematics and science instruction — through the development of teacher leaders across the 
state, through the creation of coalitions and partnerships to support reform, and through teacher 
professional development in targeted districts. The NMSI also hoped to achieve a balance 
between grassroots solutions and broader mandates: while project staff planned to develop local 
ownership and support for reform, they also sought to promote national standards and state 
frameworks, and advocate for statewide changes relating to equity, assessment, graduation 
requirements, and teacher preparation. Finally, to overcome geographical barriers, the NMSI 
looked to distance learning and technology to increase student and teacher access to resources 
and high quality instructional materials, particularly in the state’s rural, underserved 
communities. 

This case study takes a retrospective look at the Nebraska Mathematics and Science Initiative. 
Documents reviewed for this report include NMSI proposals submitted to the National Science 
Foundation (NSF); the NMSI Mid-Point Review; Abt monitoring reports; and NMSI evaluation 
reports. Telephone interviews with six persons charged with planning and implementing various 
components of the NMSI supplemented the details conveyed in these documents. Respondents 
reflected on both their achievements and the challenges they encountered in undertaking the 
systemic reform of science and mathematics education in Nebraska. This case study chronicles 
their efforts: what they accomplished, why they chose particular reform strategies, barriers that 
stood in their way, and their legacy in Nebraska’s mathematics and science education system 
today. 



Ten Years Ago: 

The Context for Mathematics and Science Education Reform in Nebraska 

The Lay of the Land 

The state of Nebraska is 500 miles across. It covers 77,000 square miles, with a sparse 
population of 1.6 million people. Approximately one-third of the state’s population lives in the 
western two-thirds of the state — a vast expanse where small, rural communities and schools can 
be separated by large distances. At the time the NMSI was funded, the average enrollment in 
about three-quarters of the state’s districts was just over 200 students. The impact on science 
and mathematics education in these systems was palpable. In grades 7-12, one teacher was often 
responsible for all science and/or mathematics classes at each grade level. While schools 
typically offered pre-calculus, chemistry, and physics courses, many teachers lacked the 
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preparation and resources for teaching them. Professional development opportunities were 
sparse, with attendance hampered by travel time. In contrast to these extreme rural systems, the 
Omaha and Lincoln school districts in the eastern part of the state — where more than 50 percent 
of the state’s population resides — deal with an array of issues common to urban school systems. 

In the early 1990s, the state’s teaching population included about 15,000 elementary teachers, 
and about 1,800 teachers of science and mathematics at the middle and high school levels. While 
Nebraska has seen a growing minority population, it remains predominantly white (over 90 
percent). At the time the NMSI was funded, the state’s minority population included four 
percent African American, two percent Latino, and about one percent Native American. 

Nebraska has struggled to diversify its teaching force, but in 1992-93, only one percent of the 
state’s public school science and mathematics teachers were members of minority groups. 

The three University of Nebraska campuses (Lincoln, Omaha, and Kearney) serve 85 percent of 
all college students attending public four- year institutions in the state. Nebraska has 17 teacher 
preparation institutions; a nationally recognized Teachers College at the University of Nebraska 
at Lincoln (UNL) provides leadership to these institutions through the Nebraska Consortium for 
the Improvement of Teacher Education. 

Local Control in a Fragmented System 

Much of what occurs in Nebraska’s schools and classrooms is guided by the state’s “independent 
pioneer spirit.” Historically, neither the state legislature nor the Nebraska Department of 
Education (NDE) has taken a directive role in mandating policies and programs. At the time the 
systemic initiative was funded, there was no statewide curriculum or assessment system. Said 
one of the NMSI Principal Investigators: “The policy landscape was one of rabid local control.” 
The large number of school districts in 1992 (about 700) reflected this “jealously guarded 
tradition” of local control. Smaller districts have fought legislative pressures to consolidate — out 
of a strong commitment to local jurisdiction, a fear of outside “education edicts,” and an 
unwillingness to give up their community identity. 

In the early 1990s, the state of Nebraska was beginning to exercise a more vocal role for itself in 
education. Legislation enacted at that time mandated a minimum of course requirements for 
school accreditation. However, in keeping with the state’s emphasis on local autonomy, the 
guidelines were relatively broad, permitting districts considerable administrative and academic 
authority in defining curriculum, assessments, and graduation requirements. Instructional 
materials continued to be adopted at the school or even individual teacher level. 

Nebraska’s Educational Service Units (ESUs) provide a regional support structure across the 
state for the widely disparate school districts. Serving as intermediate education units, the ESUs 
are a key linkage between the NDE and the districts — providing resources, information, and 
professional development. A decade ago, however, the quality of support varied widely across 
ESUs. There were “pockets of excellence,” but few districts were using Eisenhower funds, and 
there was no common vision across ESUs for professional development or curriculum. 
Opportunity and excellence depended on the proximity, experience, and inclination of resource 
people — ESU staff developers or university-based faculty — who could provide access to high 
quality staff development and instructional materials. 
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Despite efforts to provide some unity through the ESUs, the education system was 
“disaggregated and non-networked.” Across the state, science and mathematics educators rarely 
interacted. While science and mathematics professional organizations were enthusiastic and had 
a strong presence in the state, they rarely worked together. Said the NMSI Project Director: “We 
had a lot of activities in the state that were all well intended and with similar goals, but there 
wasn’t any communication.” Similarly, Department of Education science and mathematics 
consultants — stretched thin by their responsibilities in meeting needs in distant parts of the 
state — had little time or inclination for coordinating their efforts. 

The Foundations for Reform 

While these barriers posed challenges, Nebraska also had its own set of strengths to build on, 
particularly in mathematics. In 1989, the University of Nebraska-Lincoln had received Exxon 
funds to establish the Nebraska Mathematics Coalition. The statewide Coalition was “bom out 
of the belief that the system was not going to reform itself from within.” Rather, reform would 
require advocacy and leverage outside the system and at high levels in the state. 

The Coalition’s Board of Directors — an expansive group of 39 members — drew from the 
education, corporate, and policy sectors of the state, and included the Governor, state senators, 
CEOs of Nebraska’s largest corporations, college deans and presidents, and union leaders. The 
Board was charged with setting long-term goals, while an Executive Committee conducted 
fundraising and monitored progress toward meeting goals. Pushed by business leaders who were 
unhappy with the mathematics preparation of entry-level employees, UNL professors submitted 
an SSI proposal to support the goals of the Coalition. As the “sponsor” of the SSI, the Nebraska 
Mathematics Coalition held the potential to be a viable leadership alliance and a vehicle for 
identifying needs, defining goals, and strengthening networks of stakeholders with a shared 
interest in improving mathematics education. 

The state also had the advantage of strong leadership among a core group of mathematics 
professors at the University of Nebraska in Lincoln. In fact, while the Coalition gave its blessing 
to the SSI, the “intellectual epicenter” of the project was the Mathematics Department at UNL. 
The initial proposal represented a collaborative effort by three mathematics professors, two of 
whom had been working through other NSF grants to improve mathematics education. For 
example, 90 of the state’s “best” secondary mathematics teachers had participated in four years 
of teacher enhancement activities through the Nebraska Mathematics Scholars project. In 
addition, the UNL Mathematics Department had involved over 200 schools through the Junior 
Mathematics Prognosis program (JUMP), designed to increase enrollment in high school 
mathematics courses and improve the quality of these courses. With a strong cadre of lead 
teachers and school contacts across the state, faculty members saw the SSI as an opportunity to 
build on these efforts and “take it to the next level.” 

Finally, with its vast geography, Nebraska had invested considerable resources in establishing a 
viable distance learning strategy prior to the SSI. As one of the four original Public 
Broadcasting Service satellite uplinks, Nebraska ETV had been producing and distributing a 
wide variety of courses and conferences via satellite for over 10 years. Reform leaders planned 
to build on this system under the SSI, connecting grade K-12 educators through electronic 
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networks, and offering courses and resources to enhance curriculum and professional 
development opportunities for teachers across the state. 



Nebraska’ Plan for Systemic Reform 



The First Steps 

With high-level support from the Governor, Senator Bob Kerrey, and others involved with the 
Nebraska Mathematics Coalition, reform leaders designed a plan based on their assessment of 
the state context. “These were the big themes,” reflected one of the Nebraska Pis, “Enormous 
separation of people, local control at a nation-wide extreme, and the use of technology to connect 
people.” Within this context, there were undeniable needs: the mathematics curriculum was 
“excessively repetitious” at the elementary level and “rigidly compartmentalized” at the 
secondary level. Lecture and textbook-driven instruction predominated in the classroom. 
Students entering UNL were marginally prepared for college algebra. Girls and students of color 
received “subtle and persistent reinforcement” of the message that mathematics and science are 
the “domains of the white male.” 

To address these needs, the mathematics-focused proposal funded in 1991 included the following 
broad goals: (1) enhance K-12 curriculum and instruction through the development of teacher 
leaders, teacher professional development, and the implementation of exemplary materials; (2) 
increase access to high quality mathematics instruction, particularly in rural schools, through the 
development and implementation of a comprehensive distance-learning program; (3) strengthen 
the role of the Nebraska Mathematics Coalition and develop public support for higher 
achievement in mathematics and increased course offerings and graduation requirements; and (4) 
expand opportunities in mathematics for underrepresented groups through the promotion of 
culturally relevant instructional materials and equity-oriented staff development opportunities. 

While the initial proposal was designed to address significant concerns about mathematics 
education, reform leaders planned to develop closer working relationships with the science 
education community, and to fully integrate science into the initiative after the mathematics 
component had been “firmly established.” To the mathematics community, these intentions 
seemed sound at the time: the Nebraska Mathematics Coalition was well positioned politically to 
champion reform, as were UNL mathematics faculty who “knew the terrain” and possessed the 
leadership, experience, and commitment needed to launch a major reform effort. Teacher leaders 
and a nationally recognized distance learning system would help reform leaders contend with 
geographical barriers and the vast differences between rural and urban districts. These were the 
capacities the NMSI would build on. 

Augmenting the Plan 

While NSL provided funds to UNL in 1991 for mathematics education reform, the Loundation 
almost immediately suggested that reform leaders expand their vision to include science from the 
outset. Over the next two years, Nebraska devoted a significant amount of time and energy to 
engaging members of the science community, and devising a plan for a “fully systemic 
initiative.” With the funding of a second proposal in 1993, the project officially became the 
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Nebraska Mathematics and Science Initiative. Following suit, the “parent” organization became 
the Nebraska Mathematics and Science Coalition to reflect the expanded focus for reform. 

By 1993, the state was pushing its own reform agenda, with the Nebraska School Restructuring 
Commission recommending substantive changes in school management, curriculum, instruction, 
assessment, professional development, and community engagement. Outcomes-driven language 
pushed for “transformational schools.” The “High Performance Learning Model” promoted 
rigorous courses in science, mathematics, and technology. The “School Improvement Planning 
Process” promoted self-evaluation, with schools receiving technical assistance from the NDE. 

All of these efforts signaled the state’s desire to take a more directive role. 

Reform leaders hoped to work in concert with and capitalize on these efforts, using science and 
mathematics as the basis for systemic change. The state plan, however, was broad in scope, with 
little coordination and no unifying vision. In defining its niche as “catalyst” for reform, the 
NMSI sponsored a series of regional discussions that engaged over 200 teachers, scientists and 
mathematicians, NDE staff, school administrators, business and community leaders, and others. 
Through these meetings, the NMSI developed a vision for reform that encompassed quality 
learning, teacher preparation, equity, and accountability. The NMSI also sought to clarify its own 
role within the system: targeting unmet needs, for example, among underrepresented groups; 
strengthening existing efforts; and building capacity and infrastructure to sustain reform. Said 
one of the NMSI Pis: 

The original proposal understood what a small amount of money the SSI funds would be 
by comparison to the total amount spent on education in the state. Also, we realized that 
to continue expensive programs after the end of the NMSI without federal funds was 
simply not in the cards. Thus, we set about to make changes that would have a lasting 
effect on education in Nebraska. That certainly included our efforts to leverage much 
greater support for technology as a way to overcome problems caused by great amounts 
of land for the number of citizens in our state. We also had the belief that it was very 
important to change the teacher workforce itself. One of the lessons of the Nebraska 
Mathematics Scholars [project] was that helping teachers become professionals with a 
commitment to their discipline and connected to each other caused a significant change 
in the math teachers of Nebraska. We wanted to cause a similar impact on elementary 
school teachers and the science teachers of the state. 

While the augmentation proposal expanded curriculum enhancement, professional development, 
the development of lead teachers, and the integration of technology to science education, it also 
incorporated new strategies that reflected developments in the state’s reform landscape. For 
example, the NDE had received a grant from the U.S. Department of Education in 1991 to 
develop state frameworks in mathematics and science, and create curricular models based on 
these frameworks. To build on these efforts, the new NMSI proposal included a 
Frameworks/Model Programs component to promote understanding of the newly developed state 
frameworks, and provide teachers with access to existing high quality materials, as well as 
innovative materials developed by the NDE and pilot tested by Nebraska teachers. 
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The new proposal also included a Technology Integration component, which expanded the 
NMSI’s plan for providing access to experts, information, and resources through technology. 
Reform leaders would offer training on Internet use; provide rural and Native American schools 
with network access; utilize ESU staff to support teachers in the use of exemplary technology- 
based programs in science and mathematics; and facilitate communication among teachers and 
university faculty around technology-related programs and issues. 

The augmentation proposal expanded the NMSI’s equity focus with a plan for strengthening 
connections with community-based science activities. Through the Community Science 
component, the NMSI would provide training and develop links among youth leaders, informal 
science educators, and teachers. The centerpiece was to be the Nebraska Tribal Science Project. 
Working with the four Nebraska tribes — Omaha, Winnebago, Santee Sioux, and Ponca — the 
NMSI proposed developing culturally appropriate instructional materials and providing training 
for teachers and youth leaders. Finally, through the Networking Local Educational Leadership 
component, the NMSI planned to assist local leaders in promoting a climate for school and 
community change, develop relationships between university and district leaders, and provide 
regional opportunities for leaders to convene. 

The Creative Tension: Grassroots Solutions, Policy Choices 

The components added under the NMSI’s augmentation proposal broadened the reach of reform 
to include science education. Equally significant, the proposal reveals a major shift in strategy. 
In the initial design, activities were centralized under the management of the statewide 
Coalition — the “umbrella” organization for reform. In contrast, the new design emphasized a 
decentralized approach with the formation of Regional Coalitions, which would play key roles in 
supporting reform activities. The new design evolved out of a “deep understanding” of the need 
for reform strategies to better reflect contextual issues, including local control and the widely 
differing needs of Nebraska’s school systems. Reform leaders wanted to “get the impetus for 
change as close to the delivery of education” as they could. Said these former NMSI staff 
members of the rationale behind this shift: 

Historically, we are a state of rugged individualists who have had to survive in widely 
separate areas. Geographical barriers were one of the huge things we faced. People 
don ’t fully understand those barriers unless they ' ve had to deal with them. They were a 
major consideration if we were going to be systemic. And because of geographical 
barriers, you have issues of local control. The Regional Coeditions helped to deed with 
those issues. You really needed to have a level ofloccd control where there were funds to 
deal with issues that could be solved by people who best understood them. (NMSI PI) 

At that time, we were just starting to provide a bit more state aid, but districts were very 
much locally funded and locally controlled. Things were pretty decentralized. There 
were no top down mandates about much of anything, which drove NSF crazy. They 
didn 7 understand that. They didn 7 understand why we wouldn 7 just charge in and tell 
local school districts how it’s going to be. That would have sunk the whole project and 
killed the key leadership. (NMSI Project Director) 
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The seven Regional Coalitions funded by the NMSI were to be modeled after the state 
Mathematics and Science Coalition. Each group — comprised of representative community 
leaders and educators — would provide structures across the state to cultivate support, 
disseminate resources, deliver professional development, build public awareness, and match 
local reform efforts with needs. In short, the new strategy promoted decision-making at the local 
level, while also creating a regional, statewide network of reformers — “the human 
infrastructure” — guided by a common vision for mathematics and science education. 

Given the state’s decentralized system, the NMSI expected to devote a limited amount of effort 
to policy. Reform leaders planned to provide a vision for reform, and choose strategies that 
might “persuade” stakeholders of the value of reform activities, rather than pushing for 
mandates. Said one of the original Pis: “We had no notion that we could affect policy. It either 
dawned on us later or it was pushed on us. But it wasn’t clear to us at the beginning.” But the 
decision also reflected local knowledge of context and players. Said one person, “At the time the 
SSI began, if you had pushed for a statewide assessment system, you’d have been strung up.” 

Influencing the development of the state frameworks in science and mathematics, and 
disseminating these guidelines through the NMSI, was one of the explicit strategies for 
bolstering supportive policies. Other efforts to influence policy would occur more indirectly 
through the work of NMSI leaders. For example, one of the SSI proposal writers was already 
advocating for changes in university entrance requirements in mathematics at the time the SSI 
was funded. As the lead post-secondary institution in the state, UNL was in a strong position to 
influence the University of Nebraska system (and subsequently other teacher preparation 
institutions in the state). Similarly, reform leaders recognized the need for a uniform, statewide 
assessment system aligned with national standards, and would advocate for such a system 
through participation on the state’s Accountability Commission. 

The “backseat” approach to influencing policy was in juxtaposition with NMSI efforts to build 
local and regional support for reform. By the end of the grant, reform leaders expected these 
combined efforts to result in a web of support for mathematics and science education reform — a 
web that included a solid core of teacher leaders; exemplary instructional materials that reflected 
frameworks and addressed equity concerns; organizational structures to foster collaboration; and 
the participation of large numbers of educators — through technology, through professional 
development, through NMSI conferences, and through Regional Coalitions. 

Still, the notion of systemic reform would prove immensely challenging in a state where the 
entrepreneurial spirit and “rugged individualism” prevail. Said one NMSI PI: 

SSIs are team efforts. And if you ’re a star, going back and playing in the band isn ’t much 
fun. The problem was that there were excellent people out there and they were concerned 
about losing control and losing their spotlight. The thought that a lot of money would be 
dropped in one bucket with a single set of choices was problematic. Some people who 
were in those pockets of excellence were looking for dividing the big pot of money into 
little pots to give them money for the truly excellent things they had been doing. But that 
wasn ’t systemic. This wasn 7 just a lump of money to improve science and mathematics 
education around the state. It was a systemic initiative, and that was something brand 
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new. The best and the brightest who were running their own show in different locations 
were going to have to be harnessed together. 

While reform leaders were aware of these statewide barriers, they themselves faced similar 
challenges within the boundaries of the NMSI: moving outside their own domains to coordinate 
efforts — across mathematics and science, across educational institutions and organizations, 
across local and state governments, and across eastern and western parts of the state. 



The Realities of Implementation: 

Challenges, Trade-offs, Shifting Strategies 

Building Vision, Managing Reform 

The NMSI was “housed fiscally, geographically, and... psychologically at the University of 
Nebraska-Lincoln,” where mathematics faculty members who had written the proposal assumed 
PI roles. Project leaders immediately sought to expand identity and ownership by locating the 
NMSI office off-campus near both the university and the Nebraska Department of Education. 

An appointed NDE liaison, with office space in the NMSI quarters, created a formal link 
between the two entities to strengthen coordination. 

Initially, the “Executive Committee” of the Nebraska Mathematics and Science Coalition 
assumed decision-making responsibilities as the SSI “Steering Committee.” With the overlap of 
membership, reform leaders hoped to reinforce the state Coalition’s role in reform, create broad 
support for the NMSI vision for mathematics and science education, and maintain the 
commitment of high-level policymakers. From the beginning, however, the Coalition 
encountered problems. A primary role for the Coalition was fundraising for mathematics and 
science education. With the funding of the systemic initiative by NSF, the Coalition lost its 
focus. In contrast, the NMSI — a “child” of the Coalition — had a clear set of activities to pursue. 
While the Coalition was charged with managing the grant, state and business leaders exhibited 
little time or inclination to participate at the level SSI leaders had envisioned. Meeting 
attendance was low, and the role of the NMSI Steering Committee diminished considerably over 
the first two years of the project. In part, the lack of participation reflected some deeper issues 
around the Coalition. Said these persons: 

The Coalition was never really engaged enough in the grant administration nor were they 
that hands-on. I think that was the vision when they were formed that was never realized 
for a variety of reasons. They were largely a well-intentioned group of busy business and 
government leaders who needed full time staffing so they could act more in a policy and 
fundraising role. ...Plus there were some things people just didn 7 realize when they 
formed the Coalition. The intermediate education units — the ESUs — looked at the 
Coalition with some suspicion and wondered what they were up to and if they were going 
to try to usurp their power. Even the mathematics and science organizations looked at it 
and said, “ What do you think you ’re doing ? Aren 7 we doing ok ? ” So there was a lot of 
misunderstanding and concern about why that group was formed and roles were never 
really clarified. At the outset, the design was flawed. (NMSI Project Director) 




People were always saying, “Let’s make sure we understand that the Coalition is the 
actual entity, the agency, the organization, and the SSI is their project. ” We were always 
struggling to keep that identity straight and there was a lot of confusion about that. A lot 
of the public didn ’t understand. In the beginning, the Coalition didn ’t have a lot of money 
but people came together for a good cause. The SSI getting a lot of money took away 
from the Coalition ’s need to be out there committing money to continue to sustain it. In 
some ways, the SSI took on a life of its own and the Coalition shriveled up. (NMSI PI) 

Efforts to make the state Coalition and the SSI “indistinguishable” from each other, with 
common vision, goals, and leadership, stalled. In 1992, after one of the NMSI Pis became ill, a 
Project Director — formerly a classroom teacher and assistant to the PI — assumed oversight of 
both the Coalition and the NMSI. But the job of managing both was far too challenging for one 
person, and in 1993, the NMSI hired its own Project Director. There were concerns among 
NMSI leaders about splitting the two groups, but the break occurred nevertheless. The decision 
“virtually killed the Coalition.” 

The new Project Director, however, proved critical for clarifying roles, structure, and leadership 
functions in the NMSI. A former university faculty member, state senator, and assistant to the 
governor, she brought credibility and strong ties with the political community. She was also 
determined to broaden representation across the state, and increase collaboration among project 
staff with “different philosophies and competing visions.” Coming into the NMSI, she described 
the situation this way: 

There was no clear definition of roles and who was supposed to do what. The project 
was run more like a college department meeting where everyone said what they thought, 
but no one had much responsibility for doing anything. So I tried to impose structure and 
redefine roles for people, realizing that people wanted a horizontal organization, and 
that it had to be to succeed. But individuals in that group had never worked in an 
environment other than higher education. Accommodating schools and working with 
business people on the Coalition — their worlds were so different. There was no 
understanding of the political world. It was a loosely knit group of people who were all 
doing their own thing. It wasn 7 systemic, although there was a recognition that we 
needed to move forward in a more coordinated fashion. 

The 1993 science augmentation grant posed new challenges for the NMSI in terms of leadership 
and management. Changing the names of the state Coalition and the statewide initiative to be 
science-inclusive was fairly straightforward; getting the science community engaged “after the 
fact” proved to be far more difficult. While the mathematics community benefited from the 
“drive” of committed leaders, the science community had no such “champions.” Separated by 
disciplines, hindered by organizational culture, the university’s science faculty members were 
“lone rangers.” Said these NMSI staff members about trying to integrate the science community 
into the reform: 

We didn 7 involve science from the beginning, and that was a setback from the start. We 
didn 7 have the contacts in science. And the structure of the science faculty at the 
university was split between folks who didn 7 really talk to each other. There was no 
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continuous leadership. They were lone rangers. But it was a tactical error [to leave them 
out initially]. We spent an incredible amount of time trying to correct that error. But the 
science people cdways thought they were an afterthought, because they were an 
afterthought. (NMSI PI) 

The science and mathematics communities were vastly different. Science was more 
fractionated. With the augmentation proposal, it was like backing up the train or getting 
on a different track. It was difficult for the project to adjust to the change in mindset. 

They had trouble mobilizing science people. Science had no champions. In mathematics, 
there was a much broader base of support. (NMSI Evaluator) 

Mathematics people had put the SSI together and had ownership and then they had to go 
back and retrofit it. That was not a good way to start our dicdogue. (NMSI PI) 

The leadership in the science community was very fractured. Plus they had their own 
ideas about how they wanted to work. The NMSI model didn 'tfit their notions of how to 
do things. There was no science leader that could [pull] cdl the competing visions and 
styles together. Part of that is the culture of science, which [seems] very discipline- 
based, thus it struggles with collaborative efforts. . . . They were talented people who 
tended to work alone. We never recdly got the science side. We tried. We 'd go one way, 
and then we ’d go another way. We never really got there. (NMSI Project Director) 

With the funding of the science augmentation proposal, among the first steps was to expand the 
project leadership beyond the original UNL mathematics faculty members . 1 Five new Pis were 
added to the new leadership team, including science faculty, the NDE Commissioner of 
Education, and Teachers College faculty members with expertise in multicultural education. In 
addition, the new Project Director put a greater emphasis on evaluation to help determine “what 
was going on and how to change course,” and added the project evaluator to the Senior 
Leadership Team. The new structure gave the NMSI a stronger core leadership group, solid 
personal ties to the Governor and legislature through the Project Director, a stronger link with the 
NDE, broader contacts within the university community, and a recurrent source of formative 
feedback. 

Grade K-12 Teacher and Curriculum Enhancement 

The NMSI design focused heavily on teacher and curriculum enhancement. Reform leaders 
believed, however, that these components would have to be “teacher-led” if the project was to 
have any significant impact. With intensive preparation, lead teachers would play key roles as 
professional development providers. Dissemination and scale-up would occur through the 
PEERS Academy (Promoting Excellence in Education Regionally and Statewide), a series of 
workshops conducted by lead teachers in their districts and regions. Said one of the NMSI Pis: 

That was part of the message in the origincd proposal. If you recdly want to have 
sustained change, it cannot be done by a few colleagues at the university generously 
giving their time during the summers. It had to be owned by the teachers themselves. . . . 



1 One of the original Pis from the UNL mathematics department died in 1992, leaving project leadership in the hands of the two other faculty 
members. 



10 




Our point was to do more than just have summer professional development workshops. 
We wanted to promote the idea that the graduates of the program were part of a 
professional core of mathematics and science teachers in the state. They were part of the 
PEERS Academy. 

The NMSI selected lead teachers from 1 1 “partnerships,” including five individual school 
districts, two consortia of districts, and four ESUs. The rationale for identifying and cultivating 
the partnerships was two-fold: to develop teacher leadership across the state in a range of 
geographically and demographically diverse districts, and to nurture a supportive context for 
teacher professional development. The level of administrative commitment was a key factor in 
the selection of the partners. As part of the agreement to participate, districts/consortia were to 
provide lead teachers with eight release days per year to participate in NMSI activities, and 
allocate Eisenhower funds for lead teacher preparation and the workshops they would conduct 
for their colleagues. In asking districts to share financial responsibility for staff development, 
reform leaders had to overcome local perceptions that “the university was taking money from us 
again.” But according to one PI: “We held the line, with the idea that resources are limited, and 
we’ve got to give it to people who understand it and value it and are willing to pay for it.” In 
return, districts received NMSI funds to support teacher leaders and PEERS Academy 
Workshops. 

Each partnership identified four two-person teams of lead teachers. Teams represented grades 
K-3, 4-6, 7-8, and 9-12; at the secondary level, one team member represented science while the 
other represented mathematics. Two five- week summer institutes at UNL and quarterly 
Articulation Meetings during the school year helped prepare teams of lead teachers. Beginning 
with 20 K-3 lead teachers in 1992, the NMSI scaled up by grade levels in successive summers. 
Reform leaders initiated efforts at the elementary level because, in their eyes, that was “the 
battleground for educational change.” Driven by national goals and equity concerns, the NMSI 
sought to boost the comfort and confidence levels of the predominantly female elementary 
teaching force, increase the amount of time spent on mathematics and science while also 
reducing duplication of material taught, and provide students with a solid preparation for courses 
in middle and high school. 

There was “tremendous appreciation and acceptance” of NMSI staff development efforts at the 
elementary level, and the demand for PEERS Academy Workshops among grade K-6 teachers 
outstripped NMSI resources. Elementary teachers were typically “hungry” for professional 
development; added incentives included stipends and instructional materials received by each 
participant. The NMSI leveraged state lottery and Nebraska Department of Energy funds to 
expand the grade K-6 PEERS Academy Workshops to serve greater numbers of teachers. In 
several partnerships, reform leaders reported seeing “critical mass happening,” where lead 
teachers conducted PEERS Academy Workshops for every elementary teacher in the district, 
established core teams in their schools, and “brought everyone on board.” Observations by the 
NMSI monitor further revealed “a community of learners at work” during the PEERS Academy 
Workshops for classroom teachers. Said one person: “It was incredible to see. Teachers were 
talking about issues instead of the next lesson.” 
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Where there were strong lead teachers and a solid commitment from principals and 
superintendents, the partnerships worked well as a scaling up mechanism. Noted one NMSI PI: 
“The Partner districts provided the next level of leadership that allowed for scaling up. People 
that were strong went out and kept doing [PEERS Academy Workshops] well beyond the 
boundaries of the grant, with ESU support.” Still, there were obstacles to these efforts. The 
NMSI encountered some difficulty in engaging administrators as “full partners” in reform, 
despite the NMSI-sponsored conferences for administrators, school boards, and others designed 
to strengthen local awareness. Further, lead teachers were sometimes stretched thin, and cited 
the need for more support from project staff, better communication, and clarification of their 
roles. Quality control also posed problems: while project staff visited PEERS Academy 
Workshops and sometimes assisted, they too were stretched thin. As a result, professional 
development experiences varied, depending on lead teachers’ capacity for communicating key 
concepts and pedagogy. 

Providing professional development at the secondary level also challenged the NMSI. Project 
staff had difficulty recruiting middle and high school teacher leaders and filling some secondary- 
level PEERS Academy Workshops. In part, NMSI leaders blamed these difficulties on the 
chasm between science and mathematics, problems in getting lead teachers from different 
subject areas to team together, and the lack of leadership in the science community to advocate 
for participation in professional development activities. In trying to fill science workshops for 
high school teachers in particular, said one person: “There was no perceived need, and no 
champion to push it.” The mathematics-science rift had implications for elementary teachers as 
well. While the NMSI was able to involve science education faculty, content needs among 
teachers were huge. Without the involvement of university content experts in science, the NMSI 
had difficulty in meeting some of these needs. 

Despite these challenges, by all reports, the PEERS Academy Workshops had a tremendous 
impact on the teachers in the state. By the end of the NMSI, 44 teams of lead teachers had 
reached over 3,000 K-12 teachers through more than 150 10-day PEERS workshops that 
included summer sessions and school-year follow-up. Evaluations of the workshops were 
“consistently and overwhelmingly” positive: teachers reported gaining a better grasp of standards 
and frameworks, and learning ways to create an active learning environment and incorporate 
technology. Further, the NMSI evaluation team found evidence to suggest that lead teachers 
“had changed their approaches to teaching, and that some had assumed clear roles as their 
district’s ‘authorities’ in mathematics and science teaching.” 

In addition, through the PEERS Academy, the NMSI created a solid network of lead teachers 
who were active locally, regionally, and statewide — piloting instructional materials; adapting 
curricula to reflect state frameworks; conducting workshops; supporting colleagues in their 
buildings; making presentations to local school boards and at statewide professional association 
meetings; and organizing Family Science and Math nights. Finally, evaluation data suggest that 
PEERS Academy Workshops had an impact at the classroom level: participating teachers 
reported using constructivist teaching techniques, multiple assessment strategies, and 
manipulatives more frequently than non-PEERS teachers. PEERS teachers also reported more 
frequent conversations about mathematics and science education with their administrators and 
colleagues, suggesting the growth of a professional development community. 
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Technology: Getting Connected 

NMSI staff viewed technology as a tool for curriculum enhancement and professional 
development, as a vehicle for building networks, and as a source of information and resources. 
NMSI Pis’ preliminary conversations with teachers about how technology might address their 
needs pointed to applications at the “transition” points — as students moved from arithmetic to 
algebra and from high school to college mathematics. 

Three courses were developed to meet these needs. MathVantage, a video-based mathematics 
curriculum supplement with supplementary printed materials, was designed to prepare middle 
school students for the rigors of algebra and geometry; the five thematic units reflect the NCTM 
standards, as well as gender and multicultural considerations. To encourage success at the 
second transition point, the NMSI developed Practical Pre-College Mathematics ( PPCM ), 
primarily for small isolated schools with limited mathematics curricula; a two-semester 
interactive course for high school seniors, the program was initially broadcast live via satellite, 
and later made available on video. Finally, the NMSI developed Geometry for Elementary 
Teachers , a three credit-hour course delivered by satellite to increase teachers’ competence in 
teaching geometry concepts. 

Widely noted as one of the NMSI’s “solid successes,” these courses helped “move forward the 
state's commitment” to using technology as part of the education delivery system. According to 
the NMSI evaluator, the development and dissemination of the courses was highly systematic, 
with materials carefully developed, piloted, and field-tested with the active participation of 
classroom teachers, and “within the framework of diversity goals.” Both MathVantage and 
PPCM were widely used statewide ( MathVantage in over 40 percent of Nebraska’s middle level 
classrooms). Both programs have also been nationally recognized and distributed; for example, 
school districts in 38 states have purchased MathVantage for use in their middle schools. These 
programs continue to be used to enhance mathematics instruction, and have resulted in a 
valuable source of sustained funding: royalties from out-of-state purchases of both MathVantage 
and PPCM have created a “permanent endowment” that supports the work of the UNL 
mathematics department to improve mathematics education statewide. 

The second major NMSI technology-related strand was to increase teacher access to information 
and resources through their connection to and use of the Internet. While one NMSI PI noted that 
increasing Internet use seems like a “transparent” idea now, in 1990 it was a somewhat radical 
notion that NSF was reluctant to support. But reform leaders persisted as a way to deal with 
distance and separation, and the NMSI worked in concert with the NDE to provide modems and 
1-800 lines to schools. 

To a large degree, the NMSI’s emphasis on technology resulted from equity concerns. Reform 
leaders assumed that high quality distance learning courses, teamed with PEERS Academy 
Workshops and technology-related support from ESUs and lead teachers, would increase access, 
enhance instruction, and contribute to student persistence and achievement in mathematics and 
science. Said one PI: “We were trying to work on the hearts and minds of young women before 
they opted out of mathematics — to make it interesting and enjoyable so they would continue with 
it.” 
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By all reports, the “internet piece took off like crazy” and was deemed one of the NMSI’s early 
successes. Said these NMSI staff members: 

It gave teachers colleagues, especially the ones who didn ’t have anyone down the hall 
from them. They could read, contact others, and look at information from all kinds of 
sources. We don 7 have numbers on usage, but the best indication was that the initial 
modest server at the NDE was flooded and that drove the legislature to put money behind 
it and make it more accessible to teachers. (NMSI Project Director) 

Near the end of the NMSI, Nebraska was far ahead of most states in having their teacher 
workforce connected to the Internet. That was a big change. (NMSI PI) 

What came out of the SSI was teachers having computers and using the Internet. You 
cannot find a rural school now that does not have a computer. . . linked to the Internet, 
and that has changed the fabric of instruction. And much of that was due to the push for 
using technology in the SSI. We raised the consciousness, and that has helped with 
equity in rural schools. (NMSI PI) 

With access and training on Internet use provided by ESU and NMSI staff, the response by 
teachers was overwhelming. In response to lobbying efforts, the legislature expanded funding 
for technology and designated ESUs as the Internet nodes, with taxing authority to support 
technology-related implementation efforts. In short, the NMSI created a demand for services 
and the system was “forced to respond.” Said one of the Pis: “This was a field of dreams 
approach: build it and they will come. In our state, we didn’t have laws being passed that said, 
‘You must do this and that,’ so we had to do things that were persuasive.” According to the 
Project Director, the role of the NMSI was critical in these endeavors: the initiative not only 
helped to push through the legislation, more importantly, it provided a model that “built public 
support quickly” among a broad-based group of individuals and institutions — the NDE, the 
ESUs and others — who then collectively lobbied the legislature for fiscal support for technology 
expansion in the schools. 

Building Infrastructure: The Regional Coalitions 

Through teacher leaders, partnership districts, and technology, the NMSI hoped to create a 
support system for implementing improvements in science and mathematics. In the eyes of 
reform leaders, however, the Regional Coalitions were the key to sustaining these efforts. The 
Coalitions were to be the “lasting legacy,” resulting in a statewide network of “localized arms” to 
implement the vision of the state Coalition. A “grassroots network of networks,” the Coalitions 
brought together teachers, administrators, school board members and university faculty, as well 
as representatives from the community, business, and informal science. To the NMSI, these 
entities provided a vehicle for building public awareness, for cultivating local leadership, and for 
strengthening collaborative efforts to sustain reform. The Regional Coalitions were to work in 
concert with partnership districts to scale up reform efforts, recruiting teachers and schools 
outside the original districts to participate in PEERS Academy Workshops. 
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The NMSI designated Regional Coalitions to correspond geographically to the Department of 
Education’s 19 Educational Service Units, with each Coalition associated with a cluster of 2-3 
ESUs and a college or university. Linking the ESUs and Regional Coalitions capitalized on 
existing policies and infrastructure, formalizing these links and bringing some coherence — in 
vision, policy, and programs — to the regional structure already in place. Project staff knew the 
ESUs were “critical players they had to work with.” Said one former PI: 

We had to look at our governmental subdivisions. We were already working closely with 
the state Department of Education, and the ESUs are an extension of that. It was a 
structure that was cdready in place, so it made a lot of sense to have the Regional 
Coalitions put together by grouping the ESUs. 

The NMSI provided planning grants to Regional Coalitions to help them in their mission. RFPs 
included goals, guidelines, and expectations, but in line with the NMSI’s grassroots approach, 
the use of these funds was primarily left to the discretion of the Coalitions. Some hired directors 
and office assistants, while others utilized “donated” ESU staff to manage Coalition activities. In 
addition to funds, the NMSI provided Regional Coalitions with technical assistance in 
organization, collaboration, and planning to help them in their work. Assistance occurred 
through both one-on-one interaction with Coalition directors and attendance at Coalition 
meetings, which occurred monthly. Flexibility was key in these endeavors. Said one person: 
“We had to let the local leadership figure out what they wanted to do and then try to pull them 
together.” Added another: “But you had to do that gingerly.” In short, the NMSI tried to build 
in ownership, while also providing vision, learning opportunities, access to information, and 
guidelines “to make it systemic.” To strengthen the notion of a “network of networks,” the 
NMSI sponsored regional conferences 3-4 times annually to enable Coalition directors to discuss 
reform strategies and share insights, and to promote collaboration across reform-minded 
individuals and groups. 

The Regional Coalitions undertook a wide array of activities: conducting needs assessments; 
publicizing and supporting PEERS Academy Workshops; marketing mathematics and science 
education and programs; establishing liaisons with business, industry, and other stakeholders; 
sponsoring public awareness and equity-related events; and leveraging funds. Program 
components that had stalled under central project leadership, such as Community Science, were 
also incorporated into the work of the Regional Coalitions. Logistically, the shift made sense: 
with an awareness of needs and resources, local groups could better devise strategies for linking 
teachers, informal science, and community-based groups. Similarly, while public awareness 
began as a function of the central NMSI office, project leaders shifted this role to the Regional 
Coalitions; NMSI staff continued to disseminate project newsletters and contact newspapers, 
television, and radio with stories of statewide interest, but the majority of media coverage 
occurred regionally and locally to help generate public interest and demand for high quality 
science and mathematics programs. 

Building “healthy” Regional Coalitions, with active and broad-based support, proved to be a 
huge challenge, however. Coalitions often had difficulty embracing the NMSI vision, and 
understanding the nature of systemic reform. Reform leaders also had to contend with the 
varying capacities, priorities, and resources of the ESUs aligned with the Coalitions. Finally, the 



15 




NMSI experienced difficulties in setting a clear vision for the regional infrastructure. Said these 
NMSI staff members: 

Some individuals [in die Coalitions] did some good tilings, especially in Omaha, but the 
experience varied. It was our way to get federal funds to the grassroots level, but we 
never had a clear vision of how it would support mathematics and science education. We 
weren ’t clear statewide what needed to be done, and we gave money to the locals and 
they weren ’t sure either. (NMSI PI) 

The Regional Coalitions were the vision for sustainability and they didn ’t really work. 
There were some individuals who did good things. But the experience varied. They just 
viewed it as an extra funding source. (NMSI PI) 

Building infrastructure was one of the most important pieces of the original vision, but 
what that was supposed to look tike and how it would work was kind of hazy. But there 
was this knowledge that you had to have some way to help people work together and 
access resources and avoid duplication and move this process forward. One of the reed 
problems from the outset was that nobody had bothered to sit down and figure out how 
education is organized in the state. People really need to understand the terrain they ’re 
moving into. (NMSI Project Director) 

To some extent, the Regional Coalitions encountered the same problems as their state 
counterpart: lack of sustained commitment, limited experience in working collaboratively, and 
“too little knowledge of what science and mathematics teachers need.” Local leadership was 
also critical in determining the success of regional efforts. Often, one person carried out the 
work of the Coalition, without the type of support envisioned by the NMSI. Said one person: “It 
varied how well communities pulled together the Coalitions. If the lead person left, there was 
not broad enough ownership to keep it going.” Added another NMSI staff person: “Every 
Coalition needed someone with vision and only about half of them had it.” Finally, the lack of 
time and resources severely hampered the efforts of the Regional Coalitions. Noted the NMSI 
Project Director: 

There wasn't enough time for them to figure out how to become sustainable, so the money 
they received barely got them up and running, and ran out before they could get to be 
sustainable. Not enough front end planning went into this issue before forming and 
funding the Coalitions. People typically underestimate how long it can take to build a 
sustainable infrastructure — and how political and economic issues can derail them early 
in their formation if they come along at a time when the infrastructure is still weak. 

Without vision, leadership, and adequate resources, the Regional Coalitions had difficulty 
establishing clear roles and functions, beyond simply promoting professional development 
activities. Local leaders typically saw the NMSI as merely another “pot of money,” reflecting 
both the NMSI’s limited success in communicating the long-range vision for Coalitions to 
sustain reform, and the Coalitions’ struggles to embrace this vision. Said one PI about the 
Coalitions: “When the funding ran out, they ran out.” 
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Monitoring Change: The Role of Project Evaluators 

The evaluation of the NMSI was conducted by the Center for Instructional Innovation at UNL’s 
Teachers College. Initially, the evaluation plan was primarily summative in nature, consisting of 
annual surveys to gauge impact, with little attention to implementation, quality, or equity issues. 
With the hiring of the Project Director in 1993, however, the evaluation component was more 
fully integrated into the NMSI. The close proximity of evaluators on the UNL campus enabled 
them to attend core staff meetings on a regular basis, provide formative feedback on project 
activities, and play an “ongoing coaching role.” 

Given that professional development was a central strategy for reform in the NMSI, much of the 
evaluation focused on the quality and impact of these activities, with feedback provided to 
project staff on successes and areas in need of modification. Over the course of the NMSI, 
evaluators conducted numerous surveys to assess the quality of summer institutes and 
conferences; interviewed lead teachers; used teacher questionnaires and observations to gauge 
changes in teaching behaviors; and conducted case studies of participating schools. Highlights 
of evaluation results were widely disseminated through an evaluation newsletter, which kept 
NMSI staff, participants, and stakeholders informed of evaluation activities and findings. 

As the NMSI progressed, evaluators broadened their design, looking more systematically across 
key project components. For example, interviews with Regional Coalition directors provided 
project staff with critical information on the status of local efforts. Evaluators also provided 
technical assistance to Regional Coalitions in designing needs assessments and reporting data, 
and pushed the NMSI to require Coalition Directors to provide documentation of activities, 
accomplishments, and “reflections” on their efforts. The results of these products were used in 
planning conferences to help Coalition members to consider alternative strategies and reflect on 
more effective ways to use their NMSI funds. 

The evaluation was widely perceived as valuable. It “forced” NMSI staff to examine strategies 
and consider mid-course corrections. Still, assessing the impact of the NMSI in terms of equity 
and student achievement posed huge challenges, due to the lack of a statewide assessment 
system and widely varying local assessment practices. According to one person, the NMSI 
“never had reliable [student achievement] data and struggled with it all the way through,” thus 
hindering any efforts to demonstrate advances in student performance as a result of NMSI 
activities. 



What Has Changed? 
The Impact of the NMSI 



Policy and Leadership 

The NMSI took on a supportive role in the area of policy, cultivating a close relationship with 
state leaders and the Nebraska Department of Education. Building on NDE efforts, reform 
leaders were able to expand their vision and strategies, working through the ESUs and on 
frameworks, standards, and assessment committees. The NMSI was integrally involved in the 
dissemination of the frameworks (adopted in 1994) through PEERS Academy Workshops and 
statewide conferences, and was charged with co-leading a summit called by the Governor and 
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State Commissioner of Education to promote state standards and assessment. NMSI leaders also 
increased the state’s level of attention to equity by pushing for the collection of demographic 
data on students and teachers, and by supporting state legislation requiring all districts to provide 
equity and diversity training to teachers. Finally, through the joint efforts of the NMSI and the 
NDE, the state enacted legislation in 1993 requiring ESUs to provide schools with Internet 
access, and to levy taxes to pay for equipment and teacher training. All of these efforts 
demonstrated the NMSI’s capacity for collaboration and persuasion in a decentralized policy 
environment. 

Leadership in the UNL mathematics department provided a strong voice for reform during the 
NMSI grant. By the end of the project, UNL had increased college entrance requirements in 
both mathematics and science; over the course of the NMSI, high school course taking in 
mathematics rose by 16 percent, and by 17 percent in science. Said one PI: “Students coming 
into college are more prepared in algebra. The profile of the freshman class is better, and that 
implies the system as a whole is better.” Efforts to influence university entrance requirements 
succeeded primarily through the efforts of one of the NMSI Pis, who, along with other former 
NMSI staff members, remain active in pressing for education reform in the university system. 

Conferences and workshops for college and university representatives, and links with teacher 
preparation institutions through partnership districts and Regional Coalitions helped increase 
awareness of pre-service education issues, and according to project leaders, the NMSI sparked 
some collaborative efforts in the area of teacher preparation reform. For example, UNL received 
NSF funds for a joint project between the Teachers College and the UNL mathematics 
department to restructure elementary teacher preparation. Said the NMSI project director: “That 
was a direct outgrowth of the SSI. That kind of collaboration would have not happened before.” 

The NMSI was also instrumental in convincing administrators from the UNL Teachers College 
and from the College of Arts and Sciences to make a long-term commitment to strengthening 
relationships between IHEs and the K-12 system, and between content and education faculty. 

For example, the UNL-funded Center for Science, Mathematics, and Computer Education 
(CSMCE), headed by the former NMSI Project Director, continues to promote the NMSI vision, 
and remains a prominent force in science and mathematics education today — through 
professional development, grade K-12 outreach, coordination of grants across faculty and 
programs, and collaboration with the state’s professional associations. 

UNL has demonstrated its commitment in other tangible ways as well. In 1996, for example, 
mathematics/science education was designated as one of two “Areas of Strength” by the College 
of Arts and Sciences. The designation has enhanced funding to support mathematics/science 
improvement efforts — both from university and external sources — and has increased the level of 
collaboration between the mathematics and science departments. More recently, in 2000-01, 
UNL identified “Mathematics and Science Teachers for the 21st Century” as one of the 
University’s 15 academic priorities, and allocated $710,000 in “enhancement funding” over five 
years. 
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According to the Abt monitor, “Without the NMSI leadership, it is unlikely [mathematics and 
science] would have become University priorities.” One of the NMSI Pis elaborated on this 
point: 



One of the ironies is that while the SSI was focused on K-12 education, it helped cause a 
cultured change at UNL [in] how the major university in the state addresses mathematics 
and science education issues. This in turn has lead to activities like the Center for 
Science, Mathematics, and Computer Education, the Mathematics/Science Education 
Area of Strength, and the Mathematics and Science Teachers for the 21st Century, cdl of 
which are evidence of long term, lasting change at the University. Now UNL, like most 
of higher education, is going through a very difficult time fiscally and it is not clear what 
might eventually be a target for cuts, but I see these as part of a real legacy of the 
SSI... and they will have an impact on mathematics and science education for many years. 

To some degree, however, local control and geography persist in driving policy in Nebraska. For 
example, efforts by NMSI Pis to influence assessment policies in the state met with little 
success. Recommendations made to the legislature for periodic statewide assessments were 
denied, due to a strong belief that districts would fiercely oppose a state-imposed testing 
program. Still, NMSI leaders believed they had “increased the appreciation of the value of 
assessment” among schools, districts, and state-level stakeholders through PEERS Academy 
Workshops, through action research projects undertaken by teachers, and through a statewide 
assessment conference. In the Phase II proposal, project staff noted the continuing barriers of 
decentralization in the area of assessment, and the promise of change: 

Nebraska has no statewide system of assessment; however, there is interest in devising a 
system of accountability that promotes individued student achievement statewide without 
losing the strengths of our current decentralized system. The leadership for such a 
system must come from state elected officials to succeed; however, the NMSI will 
continue to provide expertise and encouragement for developing systems to measure 
student achievement. Reports from the Governor, the Commission of Education and 
business leaders indicate renewed commitment to this goal. . .NMSI is well positioned to 
assist in this effort. 

Project leaders clearly expected to devote more effort to gauging student impact during the 
proposed Phase II of the NMSI, noting: “Up to this point, most of the evaluation has focused on 
teacher change, which is a critical process outcome. Without teacher change, little student 
change could be expected. As we learn that teachers are changing the way they think about, 
teach, and assess mathematics and science, we can turn to them to learn about how best to assess 
the changes in their students’ achievement.” But without a mechanism for collecting baseline 
data during Phase I of the NMSI, and for gauging the impact of NMSI activities on students, 
project leaders were at a loss for convincing NSF and others of the impact of the reform on 
student achievement. On the other hand, the NMSI set the stage for change in the area of 
assessment in the state. Noted one person of these accomplishments: “The NDE did develop an 
innovative assessment plan for the state that incorporated the best of the top down and bottom up 
models. I think the work NMSI did with teachers and schools helped build acceptance for the 
need for assessment.” 
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Since NMSI funding ended in 1997, changing needs in the state have, to some degree, shifted the 
policy focus in mathematics and science. Said the former NMSI Project Director and now 
director of the Center for Science, Mathematics, and Computer Education: 

[The SSI vision] has been usurped by other more pressing issues, like teacher shortages 
and salary. There are the competing issues of keeping the eye on the ball that the SSI 
created in terms of raising standards and quality of instruction, versus the whole issue of 
just finding mathematics and science teachers. The climate is different today and the 
issues that are dominating the minds of people in charge of setting policy are different. 

Thus, while the CSMCE is well positioned to advocate for improvements in mathematics and 
science education, other state and local priorities have tested staff in their efforts to maintain the 
NMSI’s vision, and pulled them in conflicting directions. Nevertheless, the Center for Science, 
Mathematics, and Computer Education, as well as UNL’s continuing support through other 
initiatives, represent a key legacy of the NMSI, and evidence of commitment by the university 
system to improving mathematics and science education. 

Districts and Schools 

The NMSI directed much of its resources toward improving curriculum and instruction through 
technology and professional development. Intensive preparation resulted in a strong cadre of 
teacher leaders across the state, many of whom are still active in their schools and districts. 
According to the NMSI evaluation, summer institutes for lead teachers produced “measurable 
changes” in lead teachers’ views about teaching, and in their capacity for teaching mathematics 
and science. By 1997, lead teachers were in place in 71 of the state’s 93 counties “to serve as 
catalysts” for improved instruction. 

Project leaders estimated that 45 percent of all middle and high school teachers and 17 percent of 
all elementary teachers — over 3,000 teachers in all — had participated in PEERS Academy 
Workshops by the end of the NMSI. Evaluation data suggest positive changes in PEERS 
teachers’ knowledge, beliefs, and attitudes about what constitutes good instruction. The impact 
on teachers was most evident at the elementary level: participants reported spending more time 
on mathematics and science instruction than pre-PEERS and non-PEERS teachers, and more 
frequent use of constructivist teaching techniques, hands-on activities, cooperative learning, 
multiple assessments, and equity considerations in their planning. 

The NMSI never wavered from its belief that technology could enhance instruction and build a 
community of learners statewide. Free of charge to Nebraska’s teachers during the NMSI, 
MathVantage and PPCM were widely distributed across the state, helping to achieve equity 
goals for increasing access to high quality curricula in isolated schools and among 
underrepresented groups. According to the NMSI evaluation, students who took the PPCM 
course gained significantly on their University of Nebraska mathematics placement examination 
scores. Both MathVantage and PPCM continue to be used in the state and nationally, providing 
ongoing fiscal support through royalties to mathematics education. Internet connections, 
initiated by the NMSI and institutionalized with over $13 million earmarked by the state 
legislature for completing statewide connections, enabled over 80 percent of the state’s teachers 
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to tap science and mathematics resources electronically by the end of the project. Teachers used 
the Internet for peer support, for locating resources, for communicating with lead teachers and 
university faculty, and for planning and articulation. 

While the NMSI expected to enhance classroom instruction through the dissemination of 
materials, professional development, and technology, they also hoped to build broad support for 
scaling up and sustaining these efforts. Some districts built a core of lead teachers and achieved 
critical mass, with large numbers of teachers participating over time in a continuous series of 
PEERS Academy Workshops, supported with NMSI and Eisenhower funds. For the most part, 
however, efforts to establish district and regional structures met with limited success, and little is 
left of these entities. Still, the emergence of new leaders through these structures broadened the 
base of support in the state. Lead teachers, Coalition directors, and former NMSI staff continue 
to push for improvements in science and mathematics through positions of influence — in ESUs, 
in districts, in universities, and in the state’s science and mathematics professional associations. 

NMSI efforts to advance equity goals had mixed results. While the project helped reduce 
“disparities of resources and opportunities” through distance learning and high quality 
professional development offered in “neglected” parts of the state, other equity-focused efforts 
fell short. For example, NMSI conferences in 1992 and 1993 addressed such topics as equity in 
the classroom, school-community partnerships, and multicultural instructional materials, but 
another conference was cancelled due to low registration, and programs designed to attract 
parents of minority children were poorly attended. Further, while the NMSI “attacked notions of 
gender bias directly,” project leaders were “baffled by the racial/ethnic piece,” and were 
seriously challenged in their work on the Nebraska Tribal Science Project. Said one person: 

We learned how tribes viewed the dominant white population trying to come in and tell 
them how to do things. There was a real resistance. They had had it with the latest 
university professor making them a case study. We tried to work with the tribcd leaders to 
develop quality science offerings and still respect the culture. But we were dumb about 
tribcd politics. New leaders came in. None of us would say we know how to do this. 

On the other hand, through PEERS Academy Workshops, lead teachers worked with school 
districts with high minority enrollments, reaching teachers who served 79 percent of the state’s 
minority students. Further, NMSI leaders described the NSF-funded Project Banneker in 
Omaha — aimed at improving instruction and achievement among African American 
students — as a direct outgrowth of the NMSI’s efforts to raise awareness about equity issues in 
Nebraska’s urban districts, and NMSI lead teachers have played significant roles in this project. 

The Challenges of Sustainability: Organizations and Individuals 

Initially, NMSI leaders had envisioned a central role for the statewide Coalition in keeping the 
focus on science and mathematics education reform. As the “umbrella organization linked to 
business,” the group was to assume responsibility for raising money to sustain reform activities 
after the NSF grant. Overshadowed by the NMSI, with uneven participation by key leaders, 
however, the Coalition was unable to hold its own, thereby dispelling “any hope of it growing 
into an entity that would sustain things.” Today, the Nebraska Mathematics and Science 
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Coalition exists on paper only. Other factors were at work as well. Noted the NMSI Project 
Director: 



The expectations ofNSF changed after the first and second year [of funding], placing 
more emphasis on data collection and accountability, and that took time away from 
Coalition building to fulfill those expectations. ...Suffice it to say that if a business 
coedition is going to work, both business and academia have to define their respective 
roles better at the outset of any partnership and draw on the strengths of each. This 
required more time than anyone involved with forming our Coedition took. 

The NMSI did, however, increase dialogue and collaboration among key players in 
education — the NDE, the university community, and others. Lead teachers recruited through the 
state’s professional associations helped to diminish turf issues, and “made the organizations 
stronger” through their involvement in NMSI leadership and professional development activities. 
The NMSI also strengthened leadership capacity in the NDE: high level staff in mathematics and 
science participated in PEERS Academy Workshops and as SSI staff persons before moving into 
their current positions. Their presence in the Department assures a continuing “voice” for the 
NMSI vision. 

Partnerships and Regional Coalitions also helped establish linkages across ESUs, districts, and 
universities, with virtually all of the state’s IHEs, as well as many private colleges, involved in 
the NMSI through these entities. Said one PI: “There’s more coordination on programs now, and 
more interaction between public schools and universities. The SSI got a lot of people talking to 
each other. They got involved with the SSI and then with each other.” By 1997, the NMSI had 
become the “force” in science and mathematics education, leveraging over 30 percent of its 
funding in the final year from external sources, including Eisenhower funds, the Nebraska, 
Department of Energy, the state legislature, and royalties from MeithVantage. 

Still, some reform leaders expressed reservations as to how well these collaborative relationships 
had been sustained over time. Said one person: “The SSI forced people to interact. It was 
painful at times. There were moments when people stormed out of meetings. But it forced 
people to come to grips with different ideas and opinions. It was a valuable process for 
communicating and working together. But I’m not sure that so much of that is still going on.” 
Ironically, in the end, some project staff came to believe that leaders, not infrastructure, held the 
power to sustain vision and activities: 

The legacy is reedly held within the people who hael this experience. The legacy is not 
reedlv held within the basic organizational changes. I'm afraid that with SSIs, that was 
the original intent — that organizations and the way of doing business would change so 
dramatically that the infrastructure would be there as people came and went. In reality, I 
think the legacy is held within the people rather than in a major lasting infrastructure. It 
has changed the way people do business and made them more connected, but when they 
leave, the legacy goes with them. The SSI changed the landscape in Nebraska. It created 
some connections and networking, but it’s by the good will and the experiences of the 
people who are in place. When they go, these things will be history. (NMSI PI) 
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The thing that came out of the SSI is the understanding and sophistication that it built in 
people trying to move this process forw’cird. Leaders are still working in some 
capacity — almost all of them. The biggest concern is turnover: how do you maintain that 
leadership? (NMSI PI) 

The biggest thing the SSI did was put a lot of money into developing people and leaders 
and the system has benefited. The SSI does not live on as an entity, but it lives on through 
individuals. (NMSI Component Coordinator) 

Still, the strategy for developing organizational structures to support reform was not without 
merit. ESUs that took a central role in Regional Coalitions benefited from their involvement. 

The NMSI expanded the vision and roles of ESU staff development directors, and increased 
awareness of frameworks, standards-based materials and instruction, and high quality staff 
development. Said one NMSI PI: “The professional development delivery system is much 
stronger now and much less variable. The Regional Coalitions helped ESUs to work 
together — weak ones and strong ones — on a regional basis.” The result has been a “strengthened 
network,” as well as a “more uniform level of excellence” in the professional development 
delivered by the ESUs today. 



Factors that Shaped the Nebraska SSI Story 

The NMSI design for systemic reform included strategies for building grassroots and system- 
wide support for improving mathematics and science education. A number of factors contributed 
to both the successes and challenges the NMSI encountered in implementing its design. 

> Reform leaders demonstrated a strong awareness of needs and resources in their 
design, but the state context presented formidable challenges. 

The “broad themes” — distance, local control, and technology to link people — guided much of the 
work of the NMSI. Partnerships and Coalitions sought to create networks, reduce isolation, and 
provide an overarching vision to give local efforts a system-wide coherence. Working closely 
with the Nebraska Department of Education, the NMSI sought to build on and strengthen 
existing structures and policies. Nevertheless, the state context challenged reform leaders in 
their efforts to build a systemic effort. The tradition of local control continues to create resistance 
to statewide assessment systems, making it difficult to gauge progress in areas of equity and 
achievement. Further, distance and separation in the state deterred collaboration and coalition 
building, making linkages established during the NMSI tenuous. In the end, reform leaders who 
had promoted strategies for building organizational support returned to the recurrent Nebraska 
themes of individualism and leadership as the essential components for nurturing reform. 

Without critical mass or supportive structures, however, the loss of these leaders posed a distinct 
threat to sustaining the work initiated by the NMSI. 

> Reform leaders encountered difficulties in moving beyond a university-based 
mathematics reform. 

The true impetus for reform came from the UNL mathematics department, presenting the NMSI 
with huge challenges from the beginning. Efforts to engage science faculty in meaningful ways 
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were largely unsuccessful, and, to some extent, weakened grade K-12 professional development 
efforts in science. In addition, major NMSI strategies — preparing teacher leaders, providing 
workshops for teachers, and using distance learning — reflected the expertise and interests of 
mathematics faculty members — limiting the NMSI’s capacity for designing a fully systemic plan 
from the start. Further, NMSI staff who were strong leaders in their own community found it 
difficult to collaborate across project components, and with other individuals and organizations 
outside their domain. To its credit, the NMSI recognized these weaknesses and expanded the 
project leadership — helping to bridge these gulfs and jumpstart project components. 
Nevertheless, by most accounts, the NMSI never fully overcame its identity as a UNL-centered, 
mathematics-focused reform effort. 

> Reform leaders sought to build capacity and infrastructure to support reform, but 
encountered difficulties in developing collaborative structures to sustain these efforts. 

Efforts to build capacity and infrastructure were central to the NMSI plan for scale-up and 
sustainability. Among the major successes in this area were the expansion of technology 
statewide, the use of technology for curriculum enhancement, the development of a strong cadre 
of lead teachers, and the creation of university-based policies and initiatives. Each of these 
accomplishments has enhanced the quality and level of support available for mathematics and 
science education in Nebraska. Efforts to develop local infrastructure to sustain reform were less 
successful: while Regional Coalitions and Partner Districts accomplished much during their 
tenure, their capacity for sustaining reform was hindered by a lack of leadership and resources. 

A Phase II proposal, not funded by NSF, hoped to solidify the roles of Regional Coalitions as 
entities to sustain reform; without funding, however, these entities withered away. Similarly, the 
Nebraska Mathematics and Science Coalition failed to live up to its expectations as a fundraising 
organization to sustain reform activities. Still, while the entities that the NMSI set out to create 
have dissolved, much of their work has been subsumed under the ESUs, under UNL structures 
designated for enhancing mathematics and science education, and under the work of the state’s 
science and mathematics professional associations. Inspired and strengthened by the NMSI, 
these organizations and institutions continue to push for programs that promote high quality 
mathematics and science education. 
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Appendix K 



Case Report 

Puerto Rico Statewide Systemic Initiative 




Introduction 



In 1992, the National Science Foundation funded the Puerto Rico Statewide Systemic Initiative 
(PRSSI). An alliance comprised of the Resource Center for Science and Engineering at the 
University of Puerto Rico, the Puerto Rico Department of Education, and the General Council on 
Education, the PRSSI initially received $10 million over five years. As with other statewide 
systemic initiatives, the mission of the PRSSI was ambitious: to transform the mathematics and 
science education system to one that promotes excellence in teaching and learning for all 
students. 

The strategies for achieving this mission were multi-faceted: promoting vision and supportive 
policies, developing inquiry-based instructional materials, providing professional development, 
and building structures and alliances to support reform. By the end of Phase I, the PRSSI had 
demonstrated a capacity for both vision and action in pursuing these activities, and in 1997, 
received NSF funding for an additional five years. Since then, reform leaders have expanded 
their initial strategies: strengthening infrastructure at the school and policy levels, and preparing 
teachers and administrators for roles in scaling up reform island- wide. Revamping teacher 
preparation programs, developing stronger community support, and strengthening alliances 
among a broad array of stakeholders have also been critical components under Phase II. 

The road to systemic reform is by no means a linear one, and like others engaged in these efforts, 
the PRSSI has experienced its share of detours along the way. This case study takes a 
retrospective look at what the PRSSI has accomplished, how it did so, and why it took the 
particular routes it did. Among the documents reviewed for this report are: PRSSI proposals 
submitted to NSF for Phase I and Phase II funding; Mid-Point Review and Program 
Effectiveness Reports; PRSSI annual reports; and external monitoring reports by Abt. Face-to- 
face interviews with seven persons in Puerto Rico, the majority of whom have “lived and 
breathed” the PRSSI for nearly 10 years, supplemented the details conveyed by these documents. 
As respondents reflected on their experiences with reform, they shared a behind-the-scenes 
picture of the highs and lows of “doing systemic reform.” This case study attempts to do justice 
to their story. 



In the Beginning: 

The Context For Mathematics and Science Education Reform in Puerto Rico 

The Lay of the Land 

The island of Puerto Rico covers nearly 3,500 square miles, measuring about 35 miles north to 
south and over 100 miles east to west. As a Commonwealth associated with the United States, 
Puerto Rico has its own constitution that supports a representative government; island governors 
are elected by the people and serve for a period of four years. With 3.6 million citizens, Puerto 
Rico has one of the highest population densities in the world. Nearly a third of the population 
lives in the metropolitan area of San Juan, where the tourist trade and “glitz” contrast mightily 
with the abject poverty found in the rural mountainous regions of the island. Fifty years ago, 
agriculture was the major source of income for islanders; today, manufacturing, dominated by 
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pharmaceutical and computer industries, makes up nearly 40 percent of the gross domestic 
product. 

Puerto Rico ranks as the second largest educational system in the United States, with the highest 
concentration of Hispanic students in the nation. The Puerto Rico Department of Education 
(PRDE) is charged with serving the island’s 650,000 elementary and secondary school students. 
There are over 1,600 public schools located in 100 school districts dispersed across seven 
educational regions. Over three-quarters of the students attending public schools come from low- 
income households. Private schools on the island serve approximately 200,000 students — 
typically those from middle and higher income families. Over 12,000 educators teach science 
and mathematics in grades K-12 in Puerto Rico; the vast majority of these teachers received 
their teaching degree from one of the 22 schools of education in Puerto Rico’s university system. 

Ten years ago: PRDE and the Policy Context 

By the time the PRSSI was funded in 1992, Puerto Rico had already set the stage for reform with 
the passage of Law 68. Enacted in 1990, the Organic Law of the Department of Education (Law 
68) called for a radical restructuring of the education system and gave high priority to science 
and mathematics. To be sure, the needs were glaring. Bemoaning the lack of skilled workers for 
the growing technological sector, industry representatives were pushing the Governor toward 
reform. Teachers lacked content knowledge, harbored misconceptions, and relied on lecture, 
memorization, and drills; principals and other administrators endorsed these methods as the 
“right way to teach.” 

While Law 68 mandated reform in science and mathematics education, the system was deeply 
entrenched and discouraged the kind of interaction envisioned by the PRSSI. As one Co-PI 
remembers it: 

The system was very centralized and people were talking about the need for reform. We 
all knew something had to be done to change education, particularly in science and 
mathematics. But schools were totally dependent on the Department for everything — 
materials, scheduling, resources, getting professional development going, changing the 
school organization. . . . There was just total dependence on the hierarchical structures 
that had been established. There were little boxes within the organization. People were 
distrustful of each other. The idea of working as communities or as teams even within 
schools was totally cdien. 

The passage of Law 68 suggested major changes might be afoot, but to many outside the PRDE, 
it merely signified another top-down directive in a highly centralized system. The same could be 
said for the Community Schools Law (Law 18), enacted in 1993, two years after the PRSSI was 
funded. Law 1 8 promoted site-based management, granting schools administrative and 
academic autonomy, redefining the role of central and regional PRDE personnel, and promoting 
the active participation of teachers, parents, and the community in the educational process. This 
was radical change indeed. 
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In fact, both Law 68 and Law 18 paved the way for the PRSSI, and core staff acknowledged the 
importance of timing and the “serendipity” factor. Reform leaders seized the opportunity to 
promote the PRSSI vision within this policy-friendly context: 

These were the two key elements that we used to our advantage. The reform law [Law 
68] highlighted science and mathematics... and the Community Schools Law 
decentralized power. In principle, it allowed schools to take more initiative. That was the 
key element for us. It didn ’t work out perfectly, but overall principals and teachers felt 
more at ease taking initiative. That was crucial because we started bottom up with 
reform. Without that law, we ’d have been up the creek without a paddle. (PRSSI PI) 

We’re very good at taking advantage of opportunities here — like the Community Schools 
Law. It came at a time when we were building something that could benefit from that 
law. We weren 7 straying from our path, but we used the law to our advantage. It’s 
having that sense of where you want to go and being able to pick up on cues that can help 
you get there. That has been critical. (PRSSI Co-PI) 

The fact that there was consensus among high-level stakeholders to improve science and 
mathematics education was critical as well. Said the PRSSI PI: “The Secretary of Education 
wanted it. The Governor wanted it. And we were there to push it. So that was a great 
coincidence. But this is politics and you have to take advantage when you see something.” 

With Law 68, the Governor created a new major player in the education arena — the General 
Council on Education (GCE). Charged with policymaking, accreditation, and accountability in 
the island’s K-12 schools, the GCE was to design and evaluate innovative curricula and teacher 
professional development efforts, and promote collaboration among major partners in education, 
including the PRDE, universities, schools, parents, and industry. Planners of the PRSSI saw the 
involvement of the GCE as critical to their success, and brought the Council in as a major partner 
in the alliance for systemic reform. 

Building on a Foundation 

Even before PRDE policy initiatives set a friendly stage for systemic reform, the Resource 
Center for Science and Engineering (RCSE) at the University of Puerto Rico was engaged in a 
project that would play a significant role in the design of the PRSSI. In 1989, the RCSE had 
received NSF funding for Scope, Sequence, and Coordination (SS&C), a curriculum reform 
endeavor at the intermediate level (grades 7-9). The project began creating inquiry-based 
materials that integrated science and mathematics, provided professional development for 
teachers, and worked intensively with four schools to develop teacher leaders (known as 
“Coordinators”) in science and mathematics. 

While SS&C was small in scale, it provided the structure — demonstration sites and 
Coordinators — that the PRSSI would use in designing, implementing, scaling up, and sustaining 
a larger reform effort. SS&C also allowed the staff at the RCSE to hone their own vision and 
skills; to build a team of highly committed faculty members and teachers who had a firm grasp 
of the vision; and to develop relationships of trust with both the schools and the PRDE. This was 
critical: schools saw that they were getting the support they needed (e.g., instructional materials, 
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professional development), and the PRDE saw improvements in instruction, attitudes, and 
achievement. 

If SS&C laid the foundation for the PRSSI, the Resource Center for Science and Engineering 
provided the mortar. A consortium of Puerto Rico’s major colleges and universities, the RCSE 
was well-positioned to take on the SSI: the Center serves as a regional support system for island- 
wide implementation of educational reforms, and had already pioneered numerous programs to 
improve science and mathematics education, including SS&C. From these experiences, RCSE 
staff were deeply familiar with the needs, resources, and stakeholders in the system. 

The RCSE also provided a critical link to on-going reform efforts in higher education in Puerto 
Rico. For example, the Center had developed systemic approaches to improving undergraduate 
and graduate levels of science and mathematics education through the Puerto Rico Alliance for 
Minority Participation (PR- AMP) and the Experimental Program to Stimulate Competitive 
Research (EPSCoR). These post-secondary efforts, however, were directed at a small pool of 
students exhibiting talent in science and/or mathematics. With the PRSSI, the RCSE hoped to 
expand opportunities in science and mathematics for all students, resulting in an island-wide K- 
16 + system unified in both its vision and programs. 



The Plan for Systemic Reform: 

Strategies and Expectations 

In moving the system toward a coherent vision for equitable, standards-based, high quality 
science and mathematics programs, the PRSSI planned interventions at multiple levels. Major 
goals for Phase I included: 

• Strengthening policies and partnerships to support reform and ensure systemic 
alignment and accountability; 

• Developing and implementing inquiry-based instructional materials in Spanish that 
integrated science and mathematics; 

• Building teacher capacity through intensive professional development, leadership 
opportunities, continuing education, and pre-service education reform; and 

• Building infrastructure for reform through the development of demonstration sites. 

Intrinsic to the design were several features that reform leaders viewed as critical to their 
success: a bottom-up/top-down approach, the school as the primary unit of change, and the 
incorporation of equity, evaluation, and scale-up strategies from the outset. 

The Bottom-up, Top-down Design 

The PRSSI hoped to transform the teaching and learning culture, and ensure that schools had the 
policy support they needed to implement classroom changes. School-based efforts by core staff 
would ensure that teachers received materials, professional development, and support, while the 
development of aligned policies would set high standards in science and mathematics. The 
approach was decidedly bottom-up and top-down. From the beginning, PRSSI staff saw both 
components as crucial. Said these current and former SSI staff persons: 
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When you’re dealing with such a centralized system, you have to have two things to be 
successful. One, you have to get the necessary policies in place so you have the support 
of the system, and two, precisely because ours was such a centralized system, you needed 
a bottom up approach. We have followed that approach with almost every project we 
have at the Center because we know if presidents or chancellors or Secretaries leave, 
and if we do not have solid well-prepared, well-developed teachers, the project will not 
surx’ive. If you don ’t have the top-down when it’s starting, it’s not going to fly. But if you 
don ’t have the bottom up, it’s not going to stay. You need one to get reform going, and 
you need the other for reform to remain. (RCSE Staff Member) 

We wanted to make sure we were working with the critical elements of the system. One 
of the characteristics of the PRSSI is that it has always taken a top down and bottom up 
approach so that we wouldn ’t end up with teachers who were empowered, but who didn ’t 
have the support [they needed]. So we knew we had to work with the policy changes that 
would enable that support, because we knew that the SSI wouldn ’t always be around. 
(PRSSI Co-PI) 

By design, the PRSSI addressed the top-down component by creating a formal PRSSI alliance 
consisting of the three major partners in the education system: the RCSE at the University of 
Puerto Rico, the Puerto Rico Department of Education, and the General Council on Education. 
Each of the partners was considered essential to success. And each assumed significant roles in 
the reform, working both individually and collaboratively to oversee the development of 
Standards and Frameworks, to strengthen pre-service programs and certification requirements, to 
build the capacity of teachers and schools, and to monitor the impact of reform through the 
development of accountability measures. 

The School as the Unit of Change 

The PRSSI believed aligned policies and high level support were fundamental for change to 
occur. But without school-based support, the PRSSI might well have been doomed as another 
reform initiated “from the top.” For the “bottom up” component, core staff looked to the 
schools, citing strategies in the initial design that they knew to be critical. Some of these 
strategies were based on their experiences with SS&C; others were simply “intuitive.” But the 
choice of school-level reform was a conscious decision based on the belief that “school 
empowerment” — the active involvement of administrators, teachers, parents, the community, and 
business/industry in school decision-making — was key to substantive and lasting change. 

The “whole school-based approach” was integral to this approach and deemed essential for 
building teacher and school capacity — necessary prerequisites for altering the teaching and 
learning culture. 

The whole school strategy in Phase I was critical. If I had to say one thing that we have 
done that has helped to move this thing forward, it was coming up with the idea of having 
all of the science and mathematics teachers and the principal agree that they wanted to 
do this. It forces the situation of thinking as a group and working as a team. It didn 7 
have a name at the beginning. We had talked about it, but we didn 7 name it until the 
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midpoint review. We settled on that strategy because of our experiences in SS&C — the 
need to get the buy-in to have success, to go where there’s a willingness to fix the 
problem. (PRSSI Co-PI) 

We talked about changing the culture with the whole school approach. That was an 
essential concept. All science and mathematics teachers in the school had to commit to 
reform and until they signed on, we didn ’t enter the school. If we had an advocate in the 
school, we used them to get buy in. Otherwise, you cannot change the culture. I’m 
convinced of that. You can have an outstanding teacher go back to the school, and they 
cannot influence the school unless other teachers are ready to change. (PRSSI PI) 

The PRSSI expected that by using the school as the unit of change, and by starting small, they 
could build critical mass and change the culture within participating schools, and then build a 
network of “empowered” teachers and schools for scale up. Said the PRSSI PI: “If there is one 
thing that made the culture change possible, it was that we took the school as an entity, as the 
unit of change. We took that seriously.” 

From the Outset: Scaling up, Equity, and Evaluation 

SS&C provided PRSSI staff with an early opportunity to develop a model for scaling up reform. 
Phase I called for the creation of seven intermediate schools as demonstration sites (four of 
which had been SS&C schools). These Regional Dissemination Centers (RDCs) would “drive” 
the scale-up process. Said the PRSSI PI: “The most important thing for our reform was the 
scaling up strategy — from the beginning. We used the idea of creating nuclei of reform and 
using those to cross-pollinate other schools. That’s why we started small. That was a crucial 
strategy.” The RDCs would assume major responsibilities in developing, piloting, and revising 
instructional materials, and collaborating with university faculty on providing teacher 
professional development. The Centers would also serve as model schools, advocates, and 
promoters of reform in their regions; operate as training sites for participating SSI schools; 
provide technical assistance to participating schools; and monitor quality. 

Equity was also in the plan from the beginning. In Puerto Rico, socioeconomic status is the 
defining factor in equity considerations: with 79 percent of the students in public schools 
classified as low-income, the PRSSI was assured of extending access to quality science and 
mathematics programs simply by working in these schools. But by selecting reform-ready 
schools, the PRSSI also risked working with “the best” teachers and students. To avoid this 
pitfall and to ensure that participating schools resembled the whole system — “good performers, 
as well as bad performers” — project staff used PRDE data on teacher and student populations to 
select SSI schools that were representative of other schools regionally and of the system as a 
whole. 

Finally, the PRSSI gave high priority to the evaluation of program activities from the outset. 
Said the PRSSI PI: “Assessment was always there. We knew we had to document change, 
otherwise we knew we wouldn’t change anyone.” The PRSSI also knew that evaluation had to 
be done differently: they would use data for decision-making, not for “collecting dust.” Major 
evaluation activities were conducted internally, with staff housed at the RCSE. Like other 
components, the PRSSI evaluation design had its roots in SS&C, where core staff had witnessed 
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the value of formative evaluation, based on data collected in the schools. And like other 
components initially developed under SS&C, the evaluation was labor-intensive, based on 
tracking a relatively small number of schools, teachers, and students. 

Expectations for Phase I 

The PRSSI hoped to accomplish much during its first five years. Among the 16 expected 
“measurable outcomes” listed in the Phase I proposal were: solidifying the alliance of the three 
major partners; the adoption of island-wide standards in content and assessment; the revision of 
the K-12 mathematics/science curriculum; the establishment of 21 RDCs as model schools and 
Regional Dissemination Centers; the revision of pre-service and continuing education programs; 
and the development of stronger links with industry and the community. 

The PRSSI also hoped to establish a strong presence in about 200 schools by the end of Phase I. 
The plan for working with a relatively small number of schools and building capacity gradually 
was fully intentional. But there was pressure from NSF over the small proportion of schools to 
be involved, the use of the school as the unit of change, and the scaling up strategy. With their 
knowledge of both the strengths and the challenges in the system they were seeking to change, 
PRSSI staff were adamant about keeping the design, as demonstrated by this excerpt from the 
written response to NSF after site visitors expressed concerns: 

It must be considered that as the number of schools increases, the variability of contexts 
and the complexity of the project will increase. Therefore, to increase the number of 
schools beyond 210 would put the quality of the changes in jeopardy, as it would not 
allow for quality control, and the demands placed on PRSSI staff, without a substantial 
increase in resources, would not be manageable. The progression must be orderly and 
gradual. 

Core staff elaborated in interviews: 

NSF thought we were thinking too small at the beginning, that we should have gone to 
the 1,600 schools. That would have been suicidal. We were learning as we were going 
along, and we had to make sure we fine-tuned what we were doing before we got into 
something bigger. And we wanted to build capacity gradually. It took us a lot of time to 
get people ready to understand the philosophy, to get aligned, and get moving in the 
same direction. But we had to justify [to NSF] what we were doing and why and stick to 
what we believed. We risked losing support from NSF, but it would have been a greater 
loss to cave in and do a superficial approach. 

Given the significance of the grassroots, whole school-based approach in the initial design, 
PRSSI staff reflected in interviews that, at the very least, they expected to see improvements in 
classroom instruction in participating schools, and changes in the teaching and learning culture 
by the end of Phase I. In fact, far more was accomplished, as the PRSSI laid a solid foundation 
for reform at the policy and school levels. At the same time, however, over the course of Phase 
I, the limitations posed by finite resources would become all too clear, requiring the PRSSI to set 
priorities, make choices, and adjust expectations. 
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The Realities of Implementation: 
Challenges and Trade-offs 



Building Vision and Identity 

Even with supportive policies in place (Laws 68 and 18), getting a foothold at the highest levels 
of the system initially proved to be a major challenge for reform leaders. Within the first few 
years of funding, the PRSSI experienced the turnover of four Secretaries of Education in the 
PRDE. With each change came unpredictability — new priorities, new PRDE staff, and a myriad 
of political maneuverings. It was a “total gamble.” According to several PRSSI staff, the project 
“went through many ups and downs” during that time. Over time, reform leaders became more 
adept at convincing others of the merits of the PRSSI. Said the Co-PI: “We got less frustrated, 
and learned to take it in stride. It’s thinking on your feet, and I don’t mean improvisation. It’s 
knowing what you want to do, and being willing to adjust to get things done.” The PRSSI PI 
was more to the point: “It was political skills. Anyone who doesn’t understand that reform is a 
political game doesn’t know anything about reform.” 

Building consensus around the PRSSI vision and strategies among the three major partners in the 
alliance — the RCSE, the PRDE and the General Council on Education — also called for clarity of 
purpose, patience, perseverance, and political savvy. As a “political venture,” the PRSSI 
required both “working behind the scenes” and building consensus through committees. To 
engage the three partners and other major stakeholders — university presidents, representatives of 
business and industry, PRDE staff — the PRSSI created a Steering Committee, which convened 
several times a year over the course of Phase I. The Committee was “a way to get validation and 
support.” It provided a vehicle for inclusion and collaboration, and gave participants “a sense of 
ownership.” It was also an “important social process,” providing a forum for people to voice 
opinions. But building a systemic view across a wide array of stakeholders proved challenging. 
Said these persons, each with a long history of involvement with the PRSSI: 

What people had in common was that they really wanted to make a difference in what 
they were doing. We had to work through turf issues and that took a very long time. Just 
like we say teachers need to work as a team, we needed to do that also, with the partners. 
It took a while. It took a lot of time, patience, a lot of meetings, letting people bang their 
heads against the wall. (PRSSI Co-PI) 

Sometimes you thought the vision was quite clear, but then it would revert. And it’s hard 
because you have directors from different agencies and they cdl had very strong views. 
But those meetings kept us trying to keep in tune with the vision. (Former Co-PI) 

Ten years ago, there were isolated reform efforts, like a school that took initiative, or a 
professor from the mathematics department who submitted a proposed and worked with 
two or three schools, but there was no system wide reform. From the beginning, the 
PRSSI involved cdl key players. We had mathematics and science directors, assistant 
Secretaries. Not every one supported it in the beginning. It took some convincing.... The 
most difficult part was developing a systemic view of the reform. I remember a meeting 




with one of the new Secretaries of Education and people kept talking about PRSSI, 

PRSSI, and it got to the point halfway through the meeting where I said, “This is the 
science and mathematics reform. Why don 7 we start calling it the science and 
mathematics reform of the Department of Education? Because that’s what it is. ” At that 
point, I think it clicked to them that this is not a project. (RCSE Staff Person) 

From the beginning, however, the PRSSI experienced some identity problems. While the 
initiative identified itself as a “virtual organization” representing the three partners, during the 
early years of Phase I, there was strife over roles and communication. With strong leadership 
centered at the RCSE, other partners felt less like decision-makers than observers. While all of 
the major partners wanted high quality mathematics and science programs, and while the PRSSI 
was “definitively the reform in science and mathematics” on the island, in the eyes of PRDE 
staff, it was also, to some degree, a “parallel initiative.” At the same time, current PRDE staff 
acknowledged that the Department shared the blame for these “identity” issues; longstanding turf 
issues challenged the notion of collaboration, and at times, resulted in less than adequate support 
for reform at various academic and administrative levels. Part of the problem was that the PRDE 
was having identity problems itself. With the Community Schools Law mandating site -based 
management and decentralization, PRDE administrators were trying to adjust their own roles and 
strike the “right balance.” 

Regardless of these obstacles, PRSSI staff at the RCSE knew they needed the major partners for 
sustaining reform, and throughout Phase I, core staff remained committed to doing whatever was 
needed to ensure that key players were engaged: assigning major partners tasks that “made 
sense” based on expertise and existing roles in the system; working one-on-one with the 
Governor, Secretary of Education, and others with high levels of influence; engaging 
stakeholders through committees; and biding time, if necessary, while still holding to the vision. 
By its sheer size in funding and scale, the PRSSI also carried clout in the educational landscape. 
Said the PRSSI PI: “There were other projects, but none had the scale and vision of the SSI. We 
had the advantage of the size of the funding and the vision to be truly systemic. And that helped 
everyone recognize that the SSI was the reform of science and mathematics. . .There was some 
interference that insisted they were the reform . . .and they had the power to delay some of [our] 
changes. But they didn’t have the clout to stick.” 

Perseverance in establishing the PRSSI as the “umbrella” for reform in science and mathematics 
education paid off, and the project firmly lodged itself within Puerto Rico’ s broader educational 
reform movement. Writes the Abt monitor in 1995: 

It was remarkable that [PRDE and GCE staff] for whom reform in science and 
mathematics is just one piece of a broader reform picture expressed a vision that is 
completely in harmony with the vision of the Puerto Rico SSI... Rather than saying that 
the Department of Education has adopted the SSI vision, I would say that the vision they 
have is very similar. And rather than saying that they are using the SSI vision as a guide 
to practice, I would say they are using it as part of their globed, wider efforts. 
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PRSSI Staff at the RCSE: A Leader Builds a Team 

Core PRSSI staff were housed at the RCSE and took responsibility for maintaining the vision 
and managing day-to-day decision-making. The staff’s proximity with each other allowed on- 
going informal interaction, as well as more frequent formal meetings to debrief on process and 
progress. The composition of the core team was a key factor in the PRSSI’ s success. It 
combined the strong leadership and visionary role of Manuel Gomez, Principal Investigator for 
the PRSSI, with a group of committed faculty, teachers, and evaluators who were steadfast 
“doers” — who worked in the schools and with teachers, who “did the teamwork, who supported 
people, who were there.” It was a balanced team approach, critical for both keeping the vision 
and tending to the day-to-day tasks. There was little tolerance for those who “couldn’t get things 
done.” 

Having a leader in a position to advocate for reform at the highest levels, as the PRSSI did, was 
also crucial. Gomez had a proven track record for directing high profile NSF-funded projects at 
the University of Puerto Rico, was well known outside of the university community as well, and 
was a member on the Council of Advisors to the Governor. In short, the PRSSI had a leader who 
had vision, access, and political savvy. All of these attributes were critical, particularly in the 
early years of the PRSSI as it was trying to establish itself as the reform of science and 
mathematics education. 

Among PRSSI staff and others outside the project, it is widely perceived that without the vision 
and leadership of Gomez, the PRSSI would have made little progress initially. One person at the 
RCSE reflected on NSF’s early concerns about the viability of the PRSSI without Gomez: 

NSF's question early on made perfect sense: “What’s going to happen to the PRSSI if the 
plane that Manuel is on goes down? Will the PRSSI go down too? Or are there other 
people who can take over?” At the beginning of the project, we said, “Well. ..we’re 
working on that” ’ And now we can say categorically yes. We have more experience, 
more exposure, and more mileage now. We have grown into our roles. And the way we 
play our roles and our strengths, we compliment each other. Manuel allowed leadership 
to develop. There ’s good vision and good teamwork. 

In fact, capacity building of team members was central from the beginning. Over the course of 
the PRSSI, team members were given expanded responsibilities in planning, implementation, 
and dissemination, and have subsequently assumed key leadership roles both within and across 
project components — curriculum and instruction, professional development, teacher preparation, 
and evaluation. Acquiring political savvy was an important part of this leadership development 
as well. Said Gomez: “If I dropped dead now, it would be fine. At the beginning, it would have 
never taken off. But we have built leaders now, others who have become good politicians. They 
understand very well what makes things happen.” 

Building Infrastructure, Supporting Schools 

Under Phase I, the PRSSI established Regional Dissemination Centers — “testing grounds” for 
instructional reform, school reorganization, and community involvement. The Phase I proposal 
called for increasing the number of RDCs from seven to 21 by the end of five years, including 14 
intermediate level Centers serving grade 7-9 schools, and seven grade K-6 RDCs. By the third 
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year of Phase I, RDCs were expected to be fully operational in their roles as demonstration and 
training sites. With each Center working with 10 schools, the PRSSI planned to work with about 
2,000 teachers of science and mathematics and reach over 100,000 students. Released full-time, 
RDC Science and Mathematics Coordinators were paid by the PRDE, and received intensive 
training, as well as leadership development, through teaming with university faculty on 
curriculum development and workshop facilitation. To help build learning communities of 
reform-minded educators, the PRSSI provided opportunities for working teams of Coordinators 
and teachers from RDCs to meet within and across regions. 

As planned, the PRSSI started small, piloting and revising strategies in the RDCs, and then 
disseminating new materials and practices to schools in the region served by the Center. And as 
predicted by core staff, starting small worked to their advantage. Said a former Co-PI: “It was a 
way of convincing people that this works, showing that it is successful, while at the same time, 
training these teachers so they could become trainers of others. It’s really a peer reform, and 
from that came the scale-up process. It was not possible to attack the system at the same time. 

So we started small. [Our] intention was to see how many schools we could really, really 
reform, then [have] the system support itself and continue the reforms.” 

Incentives for schools to participate in the PRSSI were substantial: tables and scientific 
equipment from the PRDE; technical assistance from the GCE on school empowerment; and 
instructional materials, professional development, and follow-up support from the RCSE. 

Teacher preparation for implementation included two summer institutes offered regionally (three 
weeks prior to implementation, one week in the summer after implementation), and eight follow- 
up sessions during the school year for two consecutive years. The PRSSI took the professional 
development to the schools — something that had never been done — and that played solidly into 
the equity focus. Said one former Co-PI: 

What we most tried to do was level the playing field in the schools, for example, in 
schools that had no materials, we provided them. And training teachers — historically, 
most of the teachers who were trained were in the metropolitan areas. So in leveling the 
playing field in teacher training, we took the professional development to the whole 
island, so that teachers in ruml areas would have access to the same quality training as 
teachers in the metropolitan areas. 

With the grassroots nature of the reform, the priorities became abundantly clear early on. 
Teachers needed instructional materials to help them understand and implement the tenets of 
reform; they also needed intensive professional development and ongoing support. Even in 
reform-ready schools, where teachers were hungry for change and motivated by the “hooks” 
provided by the PRSSI, changing beliefs and teaching behaviors was slow and labor-intensive 
work. From SS&C, core staff knew they couldn’t “overload teachers with training.” And they 
knew they had to “stay with the teachers” beyond the summer workshops, providing steady 
follow-up support. Even with a “small is better” approach, finite resources challenged the PRSSI 
as it looked toward expanding to over 200 schools by the end of Phase I. Reflected one person: 
“The follow-up helped us move forward, but it also held us back because we didn’t have enough 
people to do it. Teachers value it. They want more. It was so labor-intensive.” 
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To be sure, teachers wanted and needed as much support as they could get. Even as new schools 
were being added, RDC-based Science and Mathematics Coordinators whose role was to provide 
support to their colleagues were still themselves just “learning the ropes” and getting 
comfortable with leadership roles. Further, PRSSI staff — trying to maintain quality as more 
schools were added — had difficulty “letting go” of their school visits; it was their way of 
knowing what was happening in the classrooms and in the schools, and addressing barriers. Over 
the course of Phase I and Phase II, as Coordinators gained expertise and confidence, they 
assumed greater responsibility for monitoring quality, addressing problems, and relaying 
information to core staff. But it took time, intensive professional development, and intentional 
opportunities for team building and leadership development for the Coordinators. 

To help with these efforts, the PRSSI drew on all of the major universities island-wide, linking 
PRSSI schools with post- secondary institutions in the same region. Making these links was 
critical. Said one person: “Because of the nature of the RCSE, all of the universities were 
involved, and we were able to select the best human resources at the university level to support 
the schools. There was not a single university that was not involved as trainers in the reform.” 
Through these efforts, PRSSI staff were able to move the school culture toward one that 
promotes collaboration and continuous learning. According to core staff, this was one of their 
greatest successes. 

Still, the process was not without challenges, teacher content mastery among them. As one 
person said, “We expected that changing the culture in the schools would be one of the hardest 
things to do, but in the end it was easier than getting teachers to master all the content they didn’t 
know.” In part, the “content” dilemma stemmed from the decision to develop and provide 
schools with instructional materials — a central component during Phase I. In the early 1990s, 
prior to SSI funding, the lack of Spanish instructional materials that integrated science and 
mathematics had pushed SS&C staff in this direction; at the time the initial PRSSI plan was 
designed, the process of developing and piloting grades 7-9 modules was well underway, and it 
seemed logical to pursue this strategy. Over the course of Phase I, core staff continued honing 
these materials, and developing additional ones for grades K-6 and 10-12. 

By creating materials that exemplified the Standards, core staff believed they could more 
effectively convey to teachers their vision for curriculum and instruction. Teachers helped pilot 
and revise materials, giving them ownership and contributing to their “sense of empowerment.” 
The modules also provided an alternative to the textbook, and were used as exemplars with 
teachers in professional development. But it was a massive undertaking. In hindsight, core staff 
noted that they had severely underestimated the effort required to develop the materials, and 
described the work as a “necessary” part of the process, but “extremely difficult.” Over time, 
they saw that it was “just not worthwhile” and that resources were better spent elsewhere. 
Evaluation data alerted core staff to the need for spending more time on content in Phase II, with 
less emphasis on the modules. Working with high schools further reinforced the need for a shift 
in strategy: secondary level teachers required more flexible choices in the use of different 
curriculum models, as opposed to training in the use of specific activities. 
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Making Choices 

As “bottom-up” efforts were underway to build capacity at the school level, the PRSSI continued 
to promote the development of supportive policies and to leverage resources. Core staff 
participated on committees to review and revise Standards for Content, Assessment, and 
Professional Development; convened teams of university faculty and administrators to plan 
teacher education reform; and tapped Eisenhower funds for professional development with non- 
SSI teachers — to disseminate more widely the tenets of reform and the benefits of becoming an 
SSI school. 

These efforts remained steady, aiming for sustained impact through policy and program changes. 
The more immediate goals of supporting RDCs and schools would also have long-term 
consequences — building infrastructure — but the needs were pressing and immediate. Over the 
course of Phase I, as staff focused resources on instructional materials and professional 
development, other components became more secondary — -if not in concept, in level of attention. 
For example, while core staff encouraged schools to engage the community, parent involvement 
required considerably more effort than PRSSI staff were able to give it. In the eyes of PRSSI 
staff, teachers needed leadership development, and had to be “enlightened” on new roles parents 
might assume in the schools. It would have to be a “natural progression, a continuum that we 
built on,” based on weighing priorities, needs, and resources. Said one former Co-PI: 

We had to delay the parental involvement [component] because our system couldn ’t 
handle that many changes at one time. And our priorities were at that time developing 
materials that would be in tune with a constructivist approach and that integrated 
science and math. We prioritized by the needs we saw for teacher training, and we had 
to work with the resources we had. Sometimes the resources get stretched to a point 
where you have to make a decision: What do 1 have to do to make this more effective? 

The community is a very important part of any reform change. And in the Puerto Rican 
culture, community involvement in the schools was minimal. So we had to work the 
whole change of how the parents’ role was seen in the school. And at that juncture, that 
was going to take a lot of effort. 

Similarly, the PRSSI had to make choices about other components cited as priorities in the Phase 
I proposal, including teacher pre-service reform. Under Phase I, the project convened working 
groups of university administrators and faculty to break ground for reform — conveying vision 
and awareness of the need for change. The PRSSI funded four pilot projects to redesign content 
and methodology courses at universities responsible for the preparation of the majority of the 
islands’ teachers. But these efforts had fallen short of what core staff had envisioned. Bound by 
its own set of cultures, the university system posed formidable challenges. Said one person at the 
RCSE closely involved with pre-service reform: 

We ’re talking about different cultures. The Department of Natural Science is a culture, 
the Department of Education is a culture, teachers are another culture, and now the 
education faculty is another culture. The SSI was on the right track, but their head and 
their effort were looking somewhere else — the schools and the teachers. They thought 
teacher preparation reform would be easier to do than it was. But they underestimated 
the difficulties. 
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Faculty who were involved in the PRSSI pilot projects similarly lamented institutional and 
“process” barriers. Few had anticipated the amount of time and effort required for getting 
consensus or for the collaborative revision of pre-service courses. The process served to 
reinforce the view among PRSSI staff that teacher education reform would require substantially 
more time and effort, and that additional resources would be required. 



The Evolution of Reform: 

“Know when to hold ’em, know when to fold ’em.” 

Maintaining the Vision, Shifting the Strategies 

Phase II of the PRSSI, funded in 1997, built heavily on the work of Phase I. Major goals 
included the implementation of K-12 standards-based curriculum and instruction; scaling up 
reform to 800 schools (half of the system); institutionalizing the Regional Dissemination 
Centers; and teacher preparation reform. 

Over the course of Phase I and Phase II, the PRSSI vision never wavered, nor did the major 
strategies for achieving change. For example, teacher empowerment and the whole school-based 
approach remained central. Core staff also continued to use evaluation to document progress and 
fine-tune strategies. And equity remained deeply engrained in the design. Project staff 
continued to identify high-needs schools that were representative of the system. Phase II further 
expanded the equity focus with the inclusion of schools involved in the island's School-to-Work 
(STW) program. With the incorporation of fundamental and advanced science and mathematics 
concepts in the STW curriculum, the PRSSI hoped to improve access to quality instruction, and 
extend opportunities for higher education for STW students. 

While the overall PRSSI design remained the same, there were also shifts in strategies under 
Phase II. These adjustments signified a growing awareness of changing roles as the reform 
“matured.” For example, under Phase II the PRSSI alliance was deliberately altered: the 
“community” became one of the three major partners in reform, supplanting the General Council 
on Education. The shift was a “natural” one in the eyes of core staff. With the knowledge that 
schools could rely less and less on the PRSSI for fiscal support, the need for building greater 
community support and links between schools and industry became more pressing. Similarly, 
under Phase II, the PRSSI reconsidered its own role in reform, and looked toward making 
schools more self-sufficient — developing their capacity to seek resources through grant writing 
and community outreach. Said one Co-PI: 

In the beginning, we got the schools everything — tables and equipment — but the money 
wasn ’t there anymore, and it didn ’t make sense with our philosophy either — because they 
were always going to ask for tables. So we became less of a clearinghouse for materials 
and logistics, and shifted more focus to helping schools develop entrepreneurial skills as 
part of their empowerment, helping them to get their own resources. 

While scaling up strategies were embedded in the design from the beginning, institutionalization 
took on a greater sense of urgency under Phase II. Core staff shifted their attention to working 
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with PRDE staff to help ensure a smooth transition at the end of NSF funding. Meetings 
convened with central PRDE staff focused on how to sustain the RDCs, professional 
development needs and strategies for reaching all 1,600 schools, and the fiscal and human 
resources that would be required. The PRSSI also sought to carve out meaningful roles for 
regional PRDE administrators. Said one Co-PI: “That realization came about through working 
with the schools, and seeing that the [regional and district administrators] functioned as the 
gatekeepers. We spent a lot of time putting out fires, and finally realized that we’ve got to have 
a better strategy.” Core staff supplemented one-on-one strategies with island- wide meetings, and 
involved Science and Mathematics Coordinators to help build their capacity for working directly 
with administrators. 

Strengthening the Infrastructure 

Phase II both built on and expanded the RDC infrastructure, adding 15 new centers to the 
existing 20 for a total of 35 Centers dispersed across the island. Each Center was expected to 
serve 10 to 20 new schools, along with providing assistance to the schools already served in 
Phase I. Based on the success of the RDCs in reaching both first and second-generation schools 
during the first five years of reform, with no reported declines in quality and/or student 
performance. Phase II sought to expand the roles of the RDCs. The Centers would become 
Regional Professional Development Centers (RPDCs): deliverers of professional development in 
curriculum, instruction, school change, and parent/community involvement. In short, the RPDCs 
would move beyond their former roles as simply sites for pilot testing and dissemination, and 
would become “entrepreneurial agents of change.” The PRSSI envisioned three years to 
transform the existing RDCs into RPDCs. 

To expand the notion of systemic reform, the RPDCs would move beyond services related to 
mathematics and science to include Language Arts and other content areas. The Centers would 
also operate as sites for PRDE Title Il-funded Mathematics and Science Professional 
Development Institutes, incorporating the PRSSI vision, materials, and practices in professional 
development with teachers from non-SSI schools. And they would link with the local Urban 
Systemic Initiatives to further align reform activities. 

The new vision for the RPDCs was significant, with implications for both scaling up and 
sustaining reform. Expanded roles and responsibilities for the Centers in Title Il-funded, on- 
going professional development integrated the RPDCs more fully into the “fabric” of the PRDE, 
thereby increasing the likelihood of institutionalization. Further, the change strengthened the 
notion of the whole school-based approach by involving teachers from other disciplines. Finally, 
by cultivating entrepreneurial skills among teachers and administrators, the PRSSI increased the 
likelihood that schools could forge their own alliances and seek resources from the community. 

As the number of RPDCs increased, the PRSSI sought to move beyond critical mass at the 
school level to achieving critical mass system-wide — reaching half the schools on the island. The 
effort required considerably more qualified teachers who could function as leaders in their 
schools. As one person said: “The barrier is always personnel. We don’t have the resources like 
we did in the first five years to provide that support.” In addition, the challenges increased under 
Phase II, as later generations of schools exhibited less motivation to change than those who had 
enlisted in the reform early on. PRSSI staff and Coordinators sought out new teachers to groom 
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for leadership roles, and looked to shift responsibilities from PRSSI staff to PRDE 
administrators. But both PRSSI and PRDE staff lamented the need to speed up the process, 
pushing teachers and Coordinators into roles before they were ready, and moving reform to a 
more “superficial level.” 

Using Evaluation to Gain Credibility 

Under Phase II, the evaluation component of the PRSSI continued to be critical for tracking 
barriers, needs, and progress; for gauging the quality of professional development design and 
providers; and for measuring gains in student achievement. Feedback and interaction between 
evaluation staff and those charged with other project components was ongoing and judged to be 
instrumental in how the project refined its strategies to meet goals and objectives. 

Collecting, analyzing, and disseminating evaluation data over the course of both Phase I and 
Phase II has required substantial human and fiscal resources. Why was this component so 
heavily emphasized by the PRSSI? Said the PRSSI PI: 

“ Assessment , attribution and accountability, ” that was our mantra, although it was not 
that clear initially. Intuitively, I knew it [was important] from the beginning, but the 
articulation came out in the second phase. The weakest is the attribution. In social 
systems, the causal relationships are not linear, not simple. The best you can hope for is 
the quasi-causal relationship that is persuasive. And the emphasis is on persuasion, not 
on proving. There are too many factors. 

In short, core staff wanted to show with some degree of certainty that reform activities were 
contributing to increased student performance. To strengthen their “persuasive” powers, 
evaluation staff emphasized triangulation and the use of different measures to gauge success at 
multiple levels. Said the evaluation director: “It’s a credibility issue. For many years, we 
struggled with how to measure the value added, which is what we’re in for. Until we got a 
design, we couldn’t do that as systematically as we wanted to.” The PRSSI groomed teachers in 
the use of authentic assessments at the classroom level, and developed pre-/post tests to 
administer to students, using items adapted from NAEP and TIMMS. Pre-/post test data were 
used at the school level to provide teachers and administrators with information on student 
performance; project-wide, test results provided comparative data across SSI schools, non-SSI 
schools, and private schools, enabling the PRSSI to determine progress toward reducing 
performance gaps across student groups. 

At the system level, the PRSSI contracted with the Educational Testing Service during Phase I to 
develop a Spanish version of the National Assessment of Educational Progress. Administered by 
the GCE, the Puerto Rico Assessment of Educational Progress (PRAEP) was given to a random 
sample of 12,000 fourth and eight grade students, including PRSSI and private schools. The 
PRSSI also used PRDE assessments (though not fully aligned) to gauge improvements system- 
wide: the norm-referenced SENDA in mathematics and since 1996, the criterion referenced 
Puerto Rico Competency Test which tests both science and mathematics. 

Taken together, all of the assessment data — classroom, project level, and system-wide — showed 
clear evidence of improved student performance in SSI schools. Armed with these results, 
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PRSSI staff began to “persuade” others of the value of reform. According to one person, the 
PRDE “bought into reform because over time they saw more and more evidence that SSI schools 
were doing better than other schools.” These same results helped convince non-SSI schools of 
the benefits of participating in the reform, and helped persuade NSF that the PRSSI was having a 
positive impact on student achievement. 

The pre-/post tests assumed even greater prominence during Phase II. While PRAEP “showed 
change” and allowed the PRSSI to measure improvement, the test was extremely time- 
consuming and expensive to administer. In the end, it was also “less powerful” than the pre- 
/post tests, which provided “feedback loops” at various levels. Said one person: “The PRAEP 
tells you whether the system is doing well, but it doesn’t tell you what you need to do to 
improve. In Phase II, we wanted more of a feedback mechanism that would help us work 
towards a self-correcting system, and still be used to show progress. It’s formative feedback for 
fine-tuning the reform process.” As more schools and students participated in the reform, the 
project streamlined the process through sampling, while still providing schools with enough data 
about “where they are, how they needed to move forward, and how the PRSSI can help.” Said 
the evaluation director: 

Scaling was a big issue. We had to shift our way of looking at the world. When we were 
smaller, we could personally do a lot of the intervention. As we increased, we had to give 
up control of the pieces. The evaluation couldn ’t be done the same way anymore. 
Information was coming in faster from the Coordinators than from our people collecting 
data. And the problems were being decdt with sooner than we were getting back to 
people. So we had to restructure what we were doing. All the information we used to get 
qualitatively comes from the Coordinators now. It doesn ’t make sense for us to do that 
anymore. 

Under Phase II, the PRSSI expanded the evaluation to include “parallel assessments” for 
teachers, using student items; the test has helped core staff identify professional development 
needs and revise the design accordingly. According to PRSSI staff, adding another “more 
quantitative indicator” has helped redirect professional development and strengthened the model. 

Leveraging Resources 

Under Phase I, core staff gained a deeper understanding of “doing” systemic reform, a healthy 
respect for limited resources, and an awareness of how to apply these resources strategically. 

Said the PRSSI PI: “The resources are never commensurate with the challenge. They are 
catalytic. You detect the pressure points and use them sparingly at the right points. We’ve been 
good at that. For example, it takes $300-400,000 for the pre-/post tests, but that is money well 
spent in Phase II.” Building in an increasingly larger financial commitment by the PRDE over 
the course of Phase I and Phase II also helped free up PRSSI resources to use more strategically 
for assessment, strengthening the RPDCs, and community involvement. 

Tapping existing resources and seeking new ones continued to be critical under Phase II. For 
example, while integrating technology into the schools had been an objective under Phase I, 
logistically and electronically, the schools were ill-equipped. Using Puerto Rico’s newly 
mandated Technology Implementation Policy, the PRSSI leveraged PRDE funds for integrating 
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technology training into professional development provided by the RPDCs. Similarly, 
opportunities for PRSSI-aligned professional development were expanded through the use of 
Eisenhower funds: fully half of the island’s Title II funds (about $2 million) are currently 
assigned to the PRSSI for continuing education opportunities in grade K-12 science and 
mathematics. Over the course of Phase I and Phase II, the PRSSI also tapped $2 million from 
the industrial and private sectors for curriculum development and school support, leveraged over 
$30 million from the PRDE to support reform, and secured $5 million from universities and 
other partners for reform activities. 

Leveraging new resources for teacher education reform remained a fixed priority. During Phase 
I, pilot projects had “fertilized the earth,” built reform-minded teams of faculty and 
administrators, and served as “catalytic agents” for changes in vision, attitudes, and policy 
regarding teacher preparation. Having laid this foundation, the PRSSI was more fully primed to 
take on this component in Phase II. The strategy included continued work with university-based 
teams, but also focused on securing resources needed for overhauling the pre-service education 
system. Working both within and across institutions, committees submitted and received a 
planning grant from NSF, enabling the development of a Blueprint for Excellence in Science and 
Mathematics Teacher Preparation. Said an RCSE staff person heavily involved in the pre- 
service component: 

We owe our success a lot to the SSI. The [PRDE] was saying that this is the science and 
mathematics reform, so it was easier for me to convince the teacher preparation 
programs that we have to move in this direction because that’s the direction the 
department is moving. We have these standards, which gave us a very clear direction on 
where we needed to move. That helped. We are using more and more SSI schools as 
practicum schools. We share a common mission.... So we have used a lot of the SSI 
materials and experiences in helping us build the Collaborative. 

In 1998 (early in Phase II), the University of Puerto Rico received NSF funding for the 
Collaborative for Excellence in Teacher Preparation (CETP) to focus on pre-service education 
reform. Housed at the RCSE and staffed by those who had been involved with the PRSSI, the 
program is closely articulated with the PRSSI goals. Further, CETP utilizes PRSSI teachers and 
schools to provide mentoring, professional development, and practicum experiences for student 
teachers, further strengthening the links between schools and the university community. 



What’s Changed? 
The Impact of the PRSSI 



Policy, Infrastructure, Equity 

A decade ago, the Puerto Rico science and mathematics system was poised for major curricular 
and organizational changes. When asked how the system is different today, PRSSI staff readily 
provided a litany of improvements that support stronger science and mathematics programs. For 
example, policies promoted by the PRSSI and mandated by the PRDE have established high 
standards for curriculum and instruction. Professional Development Standards have promoted 
changes in pre-service education, continuing education, and certification requirements. Virtually 
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all of these policies reflect the PRSSI philosophy, due the involvement of core staff in their 
development and revision. While the project was less successful in putting in place system-wide 
assessment policies aligned with the PRSSI vision for instruction, project staff nonetheless 
named standards and curriculum frameworks as one of the “lasting legacies” that the PRSSI 
leaves behind. 

In addition to these policy changes, the PRSSI leaves a critical mass of schools that have 
participated in the PRSSI, and infrastructure at various levels to support reform. The PRSSI has 
developed a cadre of seasoned teacher leaders who express an awareness of the power of 
collaboration, the need for reflection, and the importance of assessment and evaluation. The 
PRSSI also leaves in place a viable structure for expanding reform island- wide. Regional 
Professional Development Centers have served as leaders, disseminators, and agents of change, 
and are firmly established within the PRDE system. They serve as the key mechanism for 
supporting schools and teachers, and for sustaining the PRSSI vision for ongoing professional 
development and community- supported science and mathematics programs. Said one PRSSI 
staff person: 

We ’ve identified what we want to be our legacies. We want to strengthen the leadership 
of the Centers — doing as much professional development for the Coordinators and 
teachers, and working with the Department of Education to [help them] understand cdl of 
this, to make decisions jointly about where we want to go — making sure that the 
materials are in shape, and ready for distribution, how the whole school strategy works, 
what the community of learners is cdl about, how we work on building cdliances with 
community and industry, tools and products for the PRDE and others. The policies are 
there. / am hopeful. 

Over the last 10 years the PRSSI has also sought to unify other programs with its vision for 
change, and leverage resources that support the mission of the PRSSI. Centered at the RCSE, 
along with the PR-AMP, EPSCoR, School-to-Work, and CETP staff, the PRSSI has extended the 
reform’s philosophical, curricular, and instructional goals through these programs, as well as 
through those centered at the PRDE (e.g., Title I and Title II). Said one person about the 
heightened level of understanding about the importance of science and mathematics education, 
“[There is] awareness among top administrators at the Department of Education that science and 
mathematics are essential to the economic development of the island.” 

Finally, PRSSI evaluation and assessment data have provided substantial evidence of the impact 
of reform on student performance and on meeting equity goals. The reform has provided access 
to high quality science and mathematics instruction and professional development in schools that 
have been historically neglected, and has used a range of measures to gauge the impact of reform 
at the classroom, project, and system levels. These data have consistently shown a narrowing of 
the performance gap between high and low SES students. 

Changes in the Teaching and Learning Culture 

The successes described above signal improvements in system-wide support for science and 
mathematics education reform. What PRSSI staff spoke about most passionately, however, 
occurred in the teaching and learning culture at the school level. Said these persons: 
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It will be very hard to do away with the changes we have made with teachers. Getting to 
the roots, the grassroots, I think we did a good job at that. We recdly turned teachers ’ 
thinking around. A new Secretary [of Education] will never get the teachers who really 
internalized this to go back to the way they were teaching before. I’m very confident that 
the curriculum philosophy will still be in place 10 to 20 years down the road. Teachers 
have mastered enough of the philosophy to structure their own lessons without being 
dependent on curriculum materials and, in the long run, that’s what recdly matters. 
(Former PRSSI Co-PI) 

Science and mathematics are more of a priority now. That awareness has been 
created — that everybody should have science and mathematics, regardless of what 
career you pursue. They are intrinsiccdly important, and the way science is 
taught — more hands-on, more conceptually oriented, with low cost materials. (RCSE 
Staff Member) 

The system is very different. We have created students who can think and analyze, we 
have given them the opportunity to do things, not memorize things. They can internalize 
and apply concepts and solve problems. I know because I have visited the classrooms. 
Teachers can explain with ease what’s happening in the classroom. There is reed 
learning taking place. We have cdso created a group of teachers who want to learn more, 
who are not satisfied with what they know. (PRSSI Curriculum Staff) 

Maybe 20 years ago, I thought we could do this in 3 or 4 years, but now I know that these 
changes need 10, 20, 30 years. It’s incredible. But these teachers have been touched 
forever. They will not go back to memorization. It touches you and it tells you this is the 
way to go, that it’s going to stay. (UPR Faculty Member) 

In short, it was change among teachers in their classrooms that reform leaders believed would 
have the most lasting impact. Not surprisingly, PRSSI staff noted that instructional 
improvements varied among teachers. Still, the impact of the project on teachers, classrooms, 
and students was widespread. By the year 2000, the PRSSI Regional Professional Development 
Centers had worked directly with 725 of the islands’ schools — almost half — including 40 percent 
of the K-6 schools, 69 percent of the intermediate level schools, and 48 percent of the high 
schools. And, as noted earlier, the PRSSI broadened its reach with teachers in both SSI and non- 
SSI schools through the use of Eisenhower funds for continuing education workshops offered 
through the RPDCs. 

Stronger Alliances 

While much of the leadership, decision-making, and work associated with the PRSSI was carried 
out at the RCSE, core staff at the university were adamant that the reform would not have been 
possible without the PRDE, which played a critical “enabling” and support role — politically and 
fiscally. Despite the perceptions of “identity” problems and “parallel” initiatives, as one person 
at the RCSE said: “No matter how hard the university might have pushed reform, without the 
support of the Department of Education, it would not have worked. That’s a key factor in 
institutionalization.” Support from the Governor and Secretaries of Education, and the 
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endorsement of the PRDE, GCE, and university system have been critical to the school and 
classroom changes that have occurred, and the policies that have been adopted. Both PRDE and 
PRSSI staff believed that the Department of Education would continue to support PRSSI- 
initiated activities, though at a reduced level. 

You have to have a model that can be sustained within the budget. The PRSSI has always 
worked with more money than the system can afford. But the process of whittling it down 
might be good because it will make it more reed — figuring out how to maintain reform 
within the reality of our budget. The Department supports the model. It’s a good 
exemplar. We can use it as a base for the work we want to do. ... The principles and the 
work of the PRSSI are very good and we need to incorporate them into our programs and 
we 're working in that direction. (PRDE Staff) 

We now have a system that has embraced reform and is ready to move forward and keep 
it going. I’m trying to be realistic, and I don ’t expect the PRDE to take over all the 
components of the SSI and keep them forever. But there are pieces of the reform that I 
feel pretty sure they will take on like the materials, focusing more on content, and the way 
we do the professional development. (PRSSI Staff) 

The PRSSI leaves much in place to sustain improvements. Still, both PRSSI and PRDE staff 
were aware that instability — at the highest levels and at the grassroots level — leaves reform 
vulnerable. While the PRSSI has developed a “blueprint for institutionalization,” there were “no 
guarantees” for the future. Said the PRSSI PI: “The question comes down to this: Is there the 
political will? It shifts. It might be there for two years and then go away. The schools are 
empowered, and that’s a stabilizing element. But there’s instability in the schools. The principal 
and teachers can shift around, and very soon the collective memory of what has happened is 
gone.” 

Still, the institutionalization of at least some of the PRSSI- initiated efforts seems likely: the 
PRDE has pledged fiscal support for sustaining the RPDCs and for expanding this model in the 
reform of other subject areas. In addition, the CEPT has the potential for aligning teacher 
preparation programs with the PRSSI vision, and for linking these programs and pre-service 
teachers with PRSSI schools. In the long run, this may be the most important legacy of the 
PRSSI: connecting key components of the system to create a continuous web of support for high 
quality science and mathematics programs for all students. Said one person at the RCSE: 

I’m convinced that the most important thing is this: you cannot work with different 
educational systems. It is one system and we all have to work together. Higher education 
has to play a major role in K-12. So the years where we have had this dichotomy in 
Puerto Rico are gone. We have to credit the SSI for uniting the K-16 system. Gomez had 
that vision long before the SSI. That’s why he kept stressing the pipeline — that we have 
to work with all levels, forging connections with universities. When the SSI became a 
possibility, it was the window of opportunity to do what we knew needed to be done. 
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Major System Changes Under the Puerto Rico Statewide Systemic Initiative 

Policy 

• Science and Mathematics Content and Assessment Standards 

• Science and Mathematics Frameworks 

• Professional Development Standards in Science and Mathematics 

• Graduation requirements for science and mathematics courses each year in grades 7-12 

• Revised teacher certification requirements emphasizing the mastery of content and skills 

Infrastructure, Articulation, and Resources 

• Regional Professional Development Centers provide professional development, support communities of 
learners, and play central roles in scaling up reform 

• Alliances between PRDE and RCSE programs (e.g., PRSSI, AMP, EPSCoR, STW, CETP, Title I, Title 
II) ensures that the vision spans the “pipeline” — K-16 and beyond 

• “Entrepreneurial,” “empowered” schools know how and when to ask for help 

• “Empowered” teachers seek out learning opportunities 

• Newly leveraged resources support teacher preparation reform and technology integration 

Access, Equity, and Achievement 

• Increased access to inquiry-based materials and instruction among low SES students 

• Increased access to high quality professional development for teachers in historically neglected schools 

• SSI students outperformed students in non-SSI schools on the PRAEP; performance gaps between 
students in SSI schools and those in private schools were reduced by half in mathematics and by a third 
in science 

• Pre-/post tests administered during Phase I and Phase II show high levels of improvement in schools with 
large concentrations of low SES students 

• Scores on College Board tests reveal that students in SSI schools out-perform those in non-SSI schools 



Factors that Shaped the Puerto Rico SSI Story 



The PRSSI’ s role was integral to improvements at the classroom, school, and system levels. The 
implementation of standards-based instructional materials, the development of the RPDCs, pre- 
service reform efforts, and stronger system-wide alliances would simply not have occurred 
without the PRSSI. A number of factors contributed to these successes: 

> Picking Up on Cues 

Reform leaders took advantage of a policy-friendly context. 

Puerto Rico had initiated broad curricular and organizational reform island-wide at the time the 
PRSSI was first funded. Without these opportunities, the PRSSI might have struggled more in 
promoting supportive policies and school-based changes. Still, credit is due to PRSSI leaders 
who saw their chance, capitalized on the opportunities, and used them to promote their vision 
and strategies. 

> Good Vision and Teamwork 

The PRSSI provided vision and leadership, and demonstrated an acute awareness of 
needs, resources, and barriers. 

Centered at the RCSE, the PRSSI was well-positioned to promote its vision and coordinate 
reform efforts across programs and institutions. Strong leaders with political savvy helped steer 
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the PRSSI around bureaucratic barriers, while a highly committed team of faculty, teachers, and 
evaluators tended to school-based needs and other system components. The PRSSI balanced 
resources against needs and priorities, articulated existing resources and leveraged new ones, 
bided time when necessary, and never lost sight of long-range goals. 

> A Continuum that We Built on 

The PRSSI built capacity and credibility through piloting, evaluating, revising, and 
expanding strategies. 

Leaders were deliberate in their choice of reform strategies — starting small, using the school as 
the unit of change, and using RDCs as “test beds.” Each step prepared the way for the next, with 
reform leaders moving as slowly and systemically as time and resources would allow, building 
capacity of core team members, major partners, and school staff along the way to ensure 
readiness for change. Evaluation data helped to gauge needs and inform the design at every 
stage, enabling core staff to build on successful strategies, and refine or abandon activities that 
failed to accomplish goals and objectives. Using data to demonstrate improvements in student 
achievement and reductions in performance gaps across student groups kept the emphasis 
squarely on equity and “value added” by reform. 

> Creating Entrepreneurial Agents of Change 

The PRSSI leaves in place structures and leaders for scaling up and sustaining reform. 

Scaling up and sustainability were integral to the PRSSI design from the beginning. PRSSI 
leaders built on earlier reform efforts, which helped propel the PRSSI vision forward in tangible 
ways through the work of Science and Mathematics Coordinators and Regional Dissemination 
Centers. The selection of reform-ready schools enabled critical mass in the primary unit of 
change. Providing “hooks” and intensive support, the PRSSI gained the confidence of teachers 
and administrators; “empowered” schools and teachers supported change within and across 
districts. Over time, Science and Mathematics Coordinators and RPDCs assumed the roles and 
responsibilities previously held by PRSSI staff. Institutionalized with PRDE support, new 
structures and leaders can support reform activities beyond the SSI grant. 

> Stressing the Pipeline 

Reform leaders built vision and supportive structures at each level of the K-16+ science 
and mathematics education system. 

The PRSSI set in motion multi-layered reform strategies, and successfully linked essential 
players in the science and mathematics education system. Working with high-level stakeholders 
and the PRDE, the PRSSI promoted vision and policies in support of high quality science and 
mathematics education. The PRSSI also forged links between the PRDE, universities, RPDCs, 
schools, and the community in support of innovative curriculum, instruction, assessment, 
professional development, and pre- service reform. Finally, the PRSSI promoted a new vision for 
teaching and learning, and created networks across schools and RPDCs to bolster opportunities 
for sustained learning communities among teachers and administrators. 
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Appendix L 



Case Report 

Vermont Statewide Systemic Initiative 




Introduction 



In 1992, Vermont was in the second cohort of states to receive funding from the National 
Science Foundation (NSF) under the Statewide Systemic Initiative program. Vermont’s SSI was 
called VISMT — the Vermont Institute for Science, Mathematics, and Technology. VISMT grew 
to be a major player in science and mathematics education reform in the state, a role that 
continues to this day. The VISMT story underscores the importance of timing, of having the 
right people in the right positions, and of remaining open to opportunities if systemic efforts are 
to move forward. 

This report examines the thinking and planning that underlay the VISMT design and 
implementation. Its focus is on the strategic approaches taken by the SSI leaders, and reasons for 
the choices they made. Primary information on VISMT activities and impacts was garnered 
from review of available documents - the Phase I and Phase II proposals to NSF, midpoint 
review and program effectiveness review reports, and reports from the external monitor and 
evaluator. Information and perspectives on the “behind the scenes” thinking and decisions were 
derived from interviews with seven persons involved in leading the initiative. Some of these 
individuals are still associated with VISMT; others were present in its earlier years but have 
since left. All, however, were involved in discussions and planning that shaped how VISMT 
began and how it evolved. They were refreshingly frank in their recollection of the times and 
their assessment of VISMT’ s impact on science and mathematics education in Vermont. 



The Vermont State Context: 

Pre-SSI 

Vermont is small and sparsely-populated. The majority of its half- mi llion residents live in small 
towns or rural areas. While the state as a whole is not considered poor, it contains large pockets 
of rural poverty. The state population is quite homogeneous, over 97 percent white. The early 
1990s saw a decline in several sectors of the state economy, impacting employment and leading 
the business sector to advocate for educational improvements to strengthen the state’s workforce. 

In 1990, Vermont had 102,000 students. Most of these attended small schools in small school 
districts. Vermont’s 384 schools were contained in 284 school districts, each governed by a 
local school board; over two-thirds of the districts served fewer than 120 students. The large 
number of districts and small size of schools meant that there was, on average, one school board 
member for every seven teachers. In fact, one SSI staff member commented that there were 
more school board members in Vermont than mathematics teachers. While the number of school 
boards was large, the administration was of more manageable size — 61 supervisory unions 
provided a superintendent and staff to manage clusters of the smaller districts. The number of 
superintendents was therefore much more manageable than the number of districts would 
suggest. 

Local autonomy is fiercely valued in Vermont, both in its communities and in its schools. 
Citizens are typically suspicious of state efforts that they perceive as trying to impose mandates 
or centralize control. Local districts were responsible for determining curriculum, adopting 
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instructional materials, approving teachers’ credentials, and providing professional development 
for their teachers. Unfortunately, in the early 1990s fiscal constraints in most districts hampered 
their ability to engage in more than minimal efforts to maintain materials and offer professional 
development. Numerous attempts to pass legislation to enhance and equalize education funding 
had been unsuccessful. 

Due to the tradition of local control, the state policy environment was not seen as a strong driver 
of education reform. This situation was beginning to change, however, under the leadership of 
the Commissioner of Education, who was a strong advocate of establishing standards and using 
them to shape the rest of the system. Several of the individuals interviewed felt that he created 
the environment that made the SSI possible. For example, a VISMT staff member commented 
as follows, 

[The Commissioner] was a very strong leader, very much liked by some and not liked by 
others. He did a great sendee to the state. Through sheer force of will he got people 
thinking systemically and statewide. That’s not easy to do in a state like Vermont that’s 
small, rural, with a strong tradition of local control. It got some people mad at him in 
the process, but it really made a difference. For the first time we had leadership that 
looked at things systemically. So the time was right when the SSI began. 

Unlike other states with a long history of state control, Vermont was in the early stages of 
transition to a more central role for the State Department of Education. The Commissioner’s 
aggressive push for standards provided a state-level platform from which VISMT could operate. 
Features of the state policy context in the early 1990s included: 

• A recently-adopted Common Core of Learning document that established the basis for 
what would eventually become the state frameworks; 

• Early development and implementation of the Vermont portfolio assessment in writing 
and mathematics. The portfolios were not mandated as a statewide accountability 
assessment, but most districts participated voluntarily; and 

• Formation of a State Professional Standards Board to consolidate authority over teacher 
licensure. 

Leadership for statewide improvement of science and mathematics education was present, but 
not particularly strong. The Department of Education was not viewed as having dynamic 
leadership in the content areas. State professional associations were active, but not visible in the 
reform arena. Leadership for science and mathematics resided in individuals from the higher 
education sector, and these tended to operate on a regional, rather than a statewide scale. Prior to 
the SSI, there were no science and mathematics projects that aimed for broad statewide impact. 
However, there were several higher education initiatives that provided people or strategies that 
were eventually brought into the SSI design. Teacher enhancement projects in physical science 
and elementary science provided summer institutes for teachers. A similar series of summer 
institutes was being conducted for mathematics teachers. And the Institute for Math Mania 
sought to bring teachers and parents together to build support for the recently released NCTM 
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Standards. Taken together, these initiatives formed what one person called a “fragmented 
mosaic”, each worthy in its own right, but not moving the state significantly forward. As a 
member of the planning team noted: 

There really wasn 7 anything gluing it all together, taking a whole systems point of view. 
There was no relationship occurring, not even conversation going on, about the 
relationship of licensing, higher ed, professional development programs, and these 
teacher groups. All were operating independently of each other. Good activities, but 
separate, taking place on their own. 

Before the SSI, then, there was little significant leadership for reform of mathematics and science 
education on a statewide level, and the state was just beginning to move toward state-level 
coordination of key elements of the system. This opened the possibility for the SSI to carve a 
niche for itself that would not impinge on established “turf.” 



Technical Strategizing: 

Developing the SSI Plan 

As a 1998 case study by SRI noted, “Traditions of local control over education are deeply rooted 
in Vermont. These traditions, which state policy-makers must respect, favor public participation 
and dialogue, consensus building, and reliance on local initiatives rather than state mandates. In 
this political and cultural context, the preferred policy instruments are persuasion, incentives, 
opportunities, and partnerships.” After an unsuccessful proposal to NSF in 1990, which was 
rejected, in part, because of a lack of stakeholder involvement, development of the SSI plan and 
proposal in 1991 followed a much more inclusive pattern. 

The Planning Process 

Under the guidance of a local superintendent who then became Deputy Commissioner, the SSI 
planners undertook a series of forums across the state to engage as many people and groups as 
they could. Discussions addressed the status of the science and mathematics education system, 
the desired outcomes of a statewide initiative, and workable mechanisms to implement the 
reforms. The project representatives interviewed felt that the process included just about 
everyone who wanted a part in the discussions: 

Participation was very broad. That’s the Vermont way. / mean we collect the same 
people together all the time to plan. We ’re a really small state. There were innumerable 
meetings about what should be in it, about the structure. A few people got mad because 
things they wanted weren 7 there, so they walked. But the inclination in the state is to 
gather folks together to really think together about what to do. If you look at some of the 
other NSF funding we’ve received, you’ll see that it’s collaborative, too. 

As a result of the discussion forums, a strong consensus was reached on the priority needs facing 
science and mathematics education in Vermont: 
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• Lack of widespread or consistent use of effective instructional strategies by 
mathematics or science teachers; 

• Inadequate and fragmented local curricula, which give little guidance about what to 
teach and how to teach it; 

• Fragmented and ineffective professional development opportunities, based on what 
individual districts have the will and the funds to support, rather than a system to 
address state- wide priorities; 

• Lack of innovative models for how schools can affect changes in their mathematics and 
science programs; 

• Inadequate teacher preparation programs and teacher licensure requirements; and 

• Lack of local infrastructure — leadership, materials, technology, community support — to 
support improvements in science and mathematics teaching. 

The SSI plan was developed to address these needs. Through the forums, planners emphasized 
building a shared vision of science and mathematics education, and sought to build the 
commitment to make the SSI happen. 

Components of the SSI Design 

The Phase I proposal put forth the goal for “all students to learn what they need in science, 
mathematics, and technology to become competent, caring, productive, responsible individuals 
and citizens, committed to continued SMT learning throughout their lives.” To achieve this goal, 
there were nine sub-goals. The SSI design addressed these through the following components, 
combining action at both the state and local levels, impacting both the policy environment and 
the grass-roots implementation of new ideas: 

Curriculum and Content Standards 

The next step at the state level in moving toward a standards-based system was to translate the 
Common Core of Learning into a set of frameworks that would guide development of local 
curricula. A former director reflected: 

The promise of standards really got our attention. The NCTM Standards were still fairly 
new. We saw that standards may be the opportunity to build some coherence into the 
system. We got excited about that. 

If standards were to be the entry point into systemic reform, as the Commissioner envisioned, 
then the SSI would need to have a significant role. Plus, it was felt that the Department of 
Education lacked the personnel and resource capacities to produce the frameworks alone. The 
SSI became a natural partner. In the words of one of the project planners: 

VISMT made a conscious decision to be an active partner in development of the statewide 
frameworks. That was very important, because they became the foundation for other 
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work in the state. And through that process VISMT was able to build some real 
credibility for its capacity in math and science. 



State Assessment 

The Vermont portfolio assessment system for writing and mathematics was in its early stages. 
The Commissioner and others at the state level envisioned a progressive state assessment system 
whose implementation would drive curriculum and instruction at the classroom level. The task 
of completing the mathematics assessment, and of developing a science assessment, would be 
difficult, time-consuming, and expensive. Again, the SSI was seen as a source of expertise and 
resources to get the job done. 

Professional Development 

The SSI planners felt that a statewide mechanism for professional development should replace 
the piecemeal approach that depended on local district initiative. With institute models in 
existing teacher enhancement projects seen as successful, the same approach was built into the 
VISMT plan — intensive summer institutes and follow-up sessions that would build teachers’ 
ability to implement inquiry-based instruction in their classrooms. Because professional 
development was still a function of local districts, this statewide approach would have to be 
voluntary. The hope (naive in retrospect) was that well-trained, enthusiastic teachers would 
return to their schools and catalyze changes in others. 

The expertise to conduct the institutes was to come from the people who had been running the 
successful efforts that existed — college and university science and mathematics educators who 
were well-respected by teachers and knowledgeable about how to structure the sessions. Placing 
the institutes under the SSI represented a recognition that a collaboration of players from various 
higher education institutions would be needed to make them successful on a statewide basis. 
There was some negative reaction from individuals who felt the funds should have come directly 
to their institutions, but this did not seem to be a major issue, and the centralization of the 
professional development activities was broadly supported in the planning team for the proposal. 

Local Leadership and Support for Reform 

State-level activity alone would not accomplish the SSI goals. VISMT designers viewed local 
action as critical to the success of the initiative. This was certainly in keeping with the grass- 
roots culture for things to happen in the state. Furthermore, designers needed to take into 
account the limited resources available to many schools. The Commissioner’s challenge grant 
program was seen as having produced some promising results in schools receiving the awards. 

As one of the original planners noted: 

We had been influenced by a challenge grant process that [ the Commissioner] had gotten 
through the legislature. He was big on providing incentives for innovation and creation, 
along with accountability for what was done. So the idea of putting some money out 
there to stimulate removal of barriers, creation of new ideas, was strong in our minds. 
There were 18 or 19 places in the state that had responded to a challenge grant a couple 
of years earlier, and they provided models of innovation. So in the first couple of years 
of VISMT that’s what we wanted to do, too — stimulate local models of innovation and 
learning from them. 



5 




The intent was to use the SSI grants to cause teams of teachers and administrators at a school to 
undertake changes in their science or mathematics programs. The process would build local 
leadership and the results would provide “existence proofs” and examples for others to consider. 
In a similar way, supporting development of local School-Community Partnerships would build 
local support for the kinds of changes envisioned by the SSI goals and, hopefully, open up 
additional resources to the schools. Like the summer institutes, the SSI was to be a catalyst and a 
source of technical assistance to enable local educators to do the right thing. 

Teacher Preparation 

The higher education piece was one of the weaker elements in the original SSI design. The plan 
called for mini-grants to support undergraduate course development, while at the same time 
working at the state level to improve the teacher licensing regulations. This component had the 
feel of an add-on, not well-integrated into the overall VISMT design. Commented a university 
person working with the project: 

Higher ed was not a strong focus for VISMT. I think the need in the state was so acute 
that attention turned in that direction, the need to address the framework and get it in 
place, to speak to the accountability concerns of the legislature. Frankly, a lot of the first 
year or more was really devoted to working on the standards, getting them in place. 

Personnel from higher education were certainly seen as important players in the SSI, but the goal 
of actually impacting preparation programs at the institutions was a loose one. 

Technology 

There is conflicting evidence on the importance of technology in the SSI design. To some, it 
was a major component, as in the following remark from a former staff member: 

We had a strong belief that information technology was a key to improvement of student 
learning. 1 don ’t think we understood what that meant at that time, we were just looking 
at getting a whole lot of hardware and software in the schools and get the schools hooked 
up. 

To others, it was almost a side-line of the core effort, with little relationship to really impacting 
teaching and learning of inquiry-based mathematics and science. The differences appear to be in 
vision for technology’s potential. Most agreed, however, that the initial focus on hardware and 
infrastructure was appropriate, given the minimal state of technology in the schools when the SSI 
began. 

Equity 

Equity was identified as a need area, not so much because it arose from the discussions, but as a 
staff member commented, because NSF expected it to be there. Initially the SSI took a fairly 
traditional approach to equity. Since the state had so few non-white students, the focus was 
placed on gender equity issues, disseminating programs and practices to create equitable learning 
environments for girls as well as boys in mathematics and science. 
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VISMT as an Independent, Non-Profit Organization 

Activities of the SSI were to be housed and operated at the Vermont Institute of Science, 
Mathematics, and Technology (VISMT). VISMT was created as an independent, nonprofit 
organization. Although the Department of Education was the official fiscal agent for the SSI, all 
funds flowed through to the VISMT office. The creation of a new entity for the SSI was widely 
supported among the planning group, although different persons gave different perspectives on 
why it was important: 

One of the big discussion items was that we looked at some of the first SSI projects and 
how they structured things. We specifically decided not to form five or six higher-ed led 
centers as an overall strategy. Instead of dividing the income of the grant among 
regional higher-ed centers, we looked at our own regions, our higher ed leaders, what we 
had to work with, and felt that was not going to get the job done. It wouldn 7 bring us 
together very well; it would keep us working separately. 

There were so many stakeholders at the table interested in working together, there was 
no reason not to keep them active. I had come out of a nonprofit background, so it was a 
natural thing for me to think about forming an organization that would keep us together 
as a group. There was no sense at any time that the Department was in charge of what 
was going to be happening. The Department was a convener, was listening, was 
matching things that it was doing with things that other people were doing. From the 
beginning, there was a sense that we would form a collaborative, that the venture was 
going to be a wide-spread statewide collaborative. It was a natural way to keep everyone 
vested in the whole process. 

One reason is that many people, particularly in the business community but also in the 
K-12 community, wanted to make sure that the organization would be fairly agile. 

Neither our higher education system nor our Department of Education is particularly 
known for their agility. It takes a long time to get things done. There was a sense of 
urgency to this, and it wouldn 7 happen in either of these two bureaucracies. There was a 
fairly strong push to make VISMT a nonprofit organization. The second factor was that 
our Department of Education in Vermont was not strong at that time. In some ways, 
compared to some states, it still is not. It was not perceived as a place to put your money. 
So the thinking was to keep it non-governmental, but also non-academic in its 
organizational structure. 

The VISMT Board of Directors was composed of 15 members representing the variety of 
stakeholders in the collaboration. It was designed as a working body, providing oversight and 
guidance to VISMT staff to establish the priorities and directions that the organization would 
take. The members also provided the links that gave VISMT a place at the table whenever a 
partner organization was dealing with issues impacting science and mathematics. At the same 
time. Board meetings provided the Pis the opportunity to ensure that members were “on the same 
page” in their vision, to reinforce their understanding of the SSI strategies, and to solidify their 
commitment to the project. 
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Technical Strategizing in VISMT Design 

The VISMT design was built on a recognition of key factors in the Vermont education system in 
the early 1990s: the history of local control of most educational decisions; the lack of local 
resources and fragmentation of previous initiatives; the emerging movement toward increased 
state influence through standards and assessments; the interest of the private and political sectors 
in improving K-12 education; and the limited presence of leadership for statewide systemic 
reform. The response of the VISMT planning group was to create a collaborative entity, outside 
any existing institution, that would combine talents and resources under a common reform 
agenda. The planned strategies were two-fold: working at the state level as the primary source of 
content area expertise in developing standards, assessments, and other policy initiatives; and 
working at the local level to enhance teacher capacity and stimulate innovative school programs. 
For the most part, the components of the SSI were set to work together in a coordinated manner. 
A few components, however, were not well-integrated into the overall design. 



Operational and Political Strategizing: 

SSI Implementation 

It is nearly ten years since VISMT’ s inception. In its operation over that time, VISMT has 
maintained the collaborative spirit that marked its founding, working as a partner with the 
Department of Education, higher education institutions, businesses, local districts, and others on 
behalf of quality mathematics and science education. Its actual work, however, has changed 
considerably from what was originally envisioned. The VISMT implementation is an illustration 
of using operational and political strategizing to chart the course of the initiative. The sections 
that follow describe various aspects of the SSI implementation during Phase I and Phase II and 
how it evolved in response to challenges and opportunities. 

Leading the SSI 

VISMT has had three Executive Directors during its existence. The first came from a university 
and engineering background. During his three- year tenure VISMT was getting off the ground. 
Much of the work was at the policy and infrastructure level, not readily visible outside of those 
involved in it. The various SSI components, while doing what they were intended to do, were 
not well-integrated with each other. The general feeling among the persons interviewed was that 
VISMT was not well-focused in its early years, at least partly as a result of limited leadership 
from the Executive Director. The first Executive Director stepped down and was replaced by the 
person who had coordinated VISMT’ s technology component. This was a major step for 
VISMT. By this time, the Pis and the Board had a better sense of what the project needed to be 
about, and could make a more informed choice of a Director to lead in that direction. As a Board 
member and a former staff member noted: 

[The second Executive Director] came in, with a highly collaborative model, believed in 
getting into the schools, getting his hands dirty. Making it happen at the school level. I 
think that was a very important step. 




[During the tenure of the second Executive Director ] there was more effort to pull people 
together, to talk with each other about what they were trying to do. Early on, they were 
acting fairly independently, but after [he] came they began to move together. 

The second Executive Director brought a new level of strategic thinking to VISMT’s daily 
operation. Under his leadership, significant changes were made to several VISMT components, 
tailoring them to the evolving state context and building on lessons learned during 
implementation. The third Executive Director, who has served for most of Phase II, has 
continued the emphasis on refining strategies based on experience and changing conditions in the 
system. A former superintendent who had been involved with VISMT since the original 
planning group, he has emphasized school-level impacts. He also has worked to establish 
VISMT’s sustainability beyond the NSF funding. The strategic decisions discussed in the 
sections that follow arose primarily during the tenure of the second and third Executive Directors 

Early on, VISMT’s Board of Directors was viewed in some quarters as over-representing the 
business and higher education sectors. The VISMT leadership worked to bring greater balance 
with the number of teachers and administrators on the Board. Although higher education was 
thought to be over-represented, the original Board roster did not include administrators from 
teacher preparation institutions. This was corrected later in the project to try to find a leverage 
point for the teacher preparation component. However, the strategy did not achieve its goal. 

Said one staff member: 

We changed the composition of the Board to get the Dean of the College of Education 
and the . . . President of the State Colleges on as members. That strategy really didn 7 
work. They come to meetings, but we don 7 see a lot of change going on in 
undergraduate education. 

Frameworks and Curriculum 

In many ways, VISMT functioned as the “research and development” arm of the Department of 
Education, providing personnel, resources, and expertise that was not available in the rather 
small Department of Education staff. This was particularly true for developing frameworks and 
assessments. The Department of Education gave VISMT the lead role on the Commission to 
develop the science, mathematics and technology section of the Vermont Framework of 
Standards and Learning Opportunities document. This represented a major boost in credibility 
for the fledgling SSI. Using the collaborative approach familiar from its own development, 
VISMT convened meetings of knowledgeable representatives from the academic, education, and 
business communities. The intent was to create a framework that integrated science, 
mathematics, and technology in a way that represented the interdisciplinary nature of most real- 
world problems. 

Unfortunately, translating the interdisciplinary vision espoused by VISMT (and the Department 
of Education) into a concrete framework proved to be difficult. A range of perspectives existed 
among Commission members about what was meant by “integrated” and “interdisciplinary.” 
Moreover, the commitment to an integrated approach (in whatever form) was not broadly shared 
in the SMT community as a whole. Early drafts of the frameworks were criticized from several 
quarters. Scientists noted the absence of what they felt were important concepts; mathematicians 
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were concerned that the mathematics was losing its identity and that some topics could not be 
addressed in an integrated manner; educators worried that the Framework would require large 
changes in their accustomed local curriculum. NSF, too, voiced concerns about the shape that 
the framework was taking. VISMT leaders worked to help NSF program officers understand 
their interdisciplinary vision, with only partial success. Pressure was applied to revise the 
Framework. 

The VISMT staff leading the Framework development tried to steer a course that maintained 
their commitment to the integrated vision while being sensitive to the concerns that were being 
voiced. A review of the draft Framework was commissioned and used with the public comments 
received to revise the Framework document. In the final draft, released in late 1995, science and 
mathematics retained their separate identities, with standards derived from the NCTM and NRC 
documents. The interdisciplinary flavor was still present, but not in the foreground as originally 
intended. The revised Framework was more acceptable, both to the broad mathematics and 
science community and to NSF. 

The Framework of Standards was adopted by the State Board of Education in January 1996. 

The SSI was nearing the end of its funding period under the original NSF award. The process 
took much longer than people had anticipated. What was originally seen as a leading element of 
the reform when the SSI began had not been realized until almost the end. Yet VISMT leaders 
acknowledged that the time taken was important, and that what might be interpreted as delays 
were in fact the result of the collaborative process, making sure that concerns were heard and 
accounted for. The Framework needed to be solid, and they were proud of the quality of the 
final product. 

Adoption of the Framework provided a new focus for VISMT professional development 
activities (discussed in more detail below). Rather than advocating for inquiry-based teaching as 
generally the “right thing to do,” VISMT could emphasize curriculum and instruction based on 
the new standards. The current Executive Director described it as follows: 

At VISMT’ s first conference, the focus was “What is inquiry? What is good practice?” 
and we stayed on that through Phase I. By the time we had the standards and statewide 
tests in place, the questions were changing — “How do you implement standards -based 
teaching?” We were beyond the awareness stage; how do you get the systems in place in 
this new, standards-based environment? 

Local districts were not mandated to shape their curriculum and instruction around the standards 
contained in the Framework, but the state assessments under development would reflect the 
standards. Thus, the Framework became a powerful leverage point for getting the attention of 
local teachers and administrators and for organizing how the professional development 
component of VISMT worked with them. 

VISMT did not stop with just looking at the impact of the standards on how local curricula ought 
to look. As the SSI was applying for Phase II support from NSF, VISMT leaders realized that 
districts would need focused assistance to implement the curriculum arising from the standards. 
In particular, they would need access to quality, standards-based instructional materials. 
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Therefore, in addition to the ongoing professional development and support for curriculum 
implementation. Phase II also saw development of regional Science Education Cooperatives. 
These were described by a staff member: 

In science, we are running a Science Education Cooperative, which serves as a 
materials sharing/distribution and professional development and support system. We ’ve 
been very intentional about building supports around the standards-based programs. 
That kind of model did not exist in Vermont 7 or 8 years ago. We ’re more mature in our 
approach to curriculum than before. . . Vermont is mostly under-resourced small 
schools. One reason for the materials network is to get economy of scale and sharing of 
resources, so all teachers have to worry about is using the material. And you can use 
the materials three times during the year, which is even more economical. Most of our 
schools are not big enough to even move kits around from class to class. Sixty percent 
of our schools might only have one or two classes at a grade level. We even have a 
partnership with UPS where they use their regular delivery trucks to move the kits out to 
the schools and back. 

VISMT’s work in the area of standards and curriculum showed a natural progression from 
development of standards, to roll-out and awareness, to support for local curriculum design and 
implementation based on the standards. 

Assessment 

At the beginning of the SSI, Vermont was in the midst of refining its portfolio assessment system 
in grades 4 and 8. Like with the Framework, VISMT provided the Department of Education 
with a source of personnel, resources, and expertise to assist in the R&D effort. VISMT took the 
lead in expanding the mathematics portfolio to the high school level, working with a statewide 
network of teachers to build capacity to score the portfolios in a reliable manner. The SSI and 
the state were stung by the criticism stemming from the Rand report on Vermont’s portfolio 
system, but were not deterred from pursuing this model of assessing student learning. Shortly 
before the Framework of Standards was completed, the State Board of Education mandated the 
mathematics portfolio assessment for all students in grades 4, 8, and 10. VISMT had made 
another significant impact on state policy. 

Believing that the portfolios were important but not sufficient, VISMT also piloted the New 
Standards Reference Exam in mathematics, which also was adopted as part of the state 
assessment system. Noted a VISMT Board member: 

The state department of education made some very good choices not to go with individual 
high-stakes testing but to go with the New Standards Reference Exam and to use that to 
drive professional development and action plans. That has really worked, it’s 
accountability done right. The project had an important role in making that happen and 
using it appropriately. 

In some circles there was a perception that VISMT was placing too much emphasis on 
mathematics in its activities, at the expense of science. In part, such a perception may have 
stemmed from the visibility of VISMT’s work on the mathematics assessments and the 
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professional development that arose from helping teachers learn to use and score the portfolios. 
The road to establishing science assessments was much more difficult. Initially, VISMT worked 
to produce an assessment that would integrate science, mathematics, and technology. As a staff 
member explained, this did not work well: 

One of the decisions in the original grant, that we have since gone away from, was to 
move to an integrated assessment in science, math, and technology, rather than having 
subject-specific assessments. That came naturally from the Common Core, and the 
Standards were written that way, science, math, and technology all together. It did not 
play out that way. The economics of a small state trying to develop its own statewide 
assessment are tough; also the way we had been working on the portfolio was used as the 
model — using large numbers of teachers to do the scoring. We learned how hard it is to 
do that. To get just one task that works, that really integrates the science, math, and 
technology, is recdly hard. We realized that we just can ’t do this, we ’ll never get the 
scale up. It was unfortunate, but it was the right decision. 

The decision was made, therefore, to work with a commercial testing company to modify an 
existing standardized science test, aligning it with the Vermont standards, which by that time 
were written for separate disciplines. 

In 1997, early in Phase II, the state legislature mandated statewide assessment in the core 
subjects, including both portfolio and standardized tests. The state-level environment had shifted 
from using assessment mainly as a tool to influence curriculum and instruction (the previous 
Commissioner’s original vision) to adding an accountability component, with data on student 
results reported and used to drive school improvement. VISMT personnel were at the table when 
the legislation was drafted. Because VISMT had a lead role in assessment development and roll- 
out, the SSI was ideally situated to help schools deal with the requirements of the assessment and 
accountability system. The manner in which they did so is described in the next sections. 

Professional Development 

Perhaps the greatest changes in strategy exhibited over the life of the SSI were in the area of 
professional development. VISMT’ s evolution in this area demonstrated both an internal 
awareness of when changes were needed and the ability to recognize and build upon 
opportunities in the changing state reform environment. As originally planned, VISMT 
professional development was built around centralized, two-week summer institutes for teams of 
teachers and administrators. The institutes focused on content and pedagogy to implement an 
inquiry approach. It was an approach modeled after the earlier teacher enhancement efforts in 
the state, and assumed quite a bit about what would happen afterward. A staff member and a PI 
recalled: 

The idea was that if we could build strong professional development, build a teaching 
force and teacher leadership that could carry the weight of the classroom 
implementation, then we might really get something done. If the conversation stayed at 
the policy and formal leadership level, then not much would happen. This is pretty 
typical of Vermont — most things that succeed are grass roots. So very early there was a 
lot of focus on building the capacity of teachers, not only to teach well but also to lead 
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the charge, if you will. One of the mistakes in those early days is we didn ’t recognize 
how important the role of formed leadership was, and in many cases we created some 
high powered teachers ready to lead in their schools and their principals or 
superintendents weren ’t on-board at all. I think we learned from that, but it was an issue 
early on. 

The centred institutes were very expensive — two weeks with residency. We knew there ’s a 
certenn tier of people who will come to a highly intensive experience, giving up the rest of 
their lives for two weeks. We were asking principals and teachers and even school board 
members to form teams and do that. 1 think you reach a saturation point with that, and 
then need to get down into folks who will come to something, but they ’re not going to 
leave family for it. How can you start to sccde up by spreading out and thinking of other 
audiences. Not just the committed, but folks you need to get their attention to get 
involved, provide access points in ways that they ’re willing to say yes. 

Participants responded well to the institutes, but participation in follow-up and school-site 
support activities was not as strong as expected. Moreover, because the state standards were still 
under development, the institutes focused on national standards, and participants sometimes 
found it difficult to make the link to local curricula. And, as noted in the quote above, VISMT 
leaders realized that scaling-up beyond the early-adopters would be difficult. The result, 
beginning in the fourth year, was to move the institutes out to regional sites (school districts, 
higher education institutions, informal science organizations) as five-day Regional Professional 
Development Institutes. The RPDIs reached another level of participants and increased 
VISMT’ s visibility at the local level. Consistency across the sites was an issue, however, both in 
terms of the content included and the quality of delivery. 

At the same time that the institutes were being moved to regional sites, another strategy was 
being implemented by VISMT. The second Executive Director was concerned that the bulk of 
the VISMT professional development activity was taking place away from the school site, and 
was therefore disconnected from what was happening in the schools. Furthermore, although the 
summer institutes were intended as leadership development experiences, teachers going back to 
their classrooms found it difficult to function effectively as reform leaders. As he explained: 

The origined model was based on a faith in the ability ofprofessioned development to 
enable teachers to go back and be leaders in their schools. What VISMT found over the 
first three or four years was that that was important, but it wasn 7 sufficient. The thought 
was that if we can bring teachers in to work in leadership capacity in VISMT for a couple 
of years, get intensive experience in content, in pedagogy, in leadership, then go back to 
their schools, they’ll have the leadership skills to move things forward. 

The result was the VISMT Teacher Associates program. Teacher Associates were outstanding 
classroom teachers recruited to work on a regional basis, providing on-site technical assistance to 
participating schools. While the Teacher Associates led workshops at the regional institutes as 
well, their primary purpose was to provide consulting and assistance to teachers in their 
classrooms and to assist school teams in working on professional development plans that better 
reflected local needs. Furthermore, the state standards were nearing completion and the VISMT 
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leadership saw them as a lever for getting schools to examine their local mathematics and 
science curricula. This became another role for the Teacher Associates. As VISMT staffers 
explained: 

[We began] regional two-day conferences on the draft Framework, since we would need 
massive professional development to help teachers understand what it meant and what it 
would mean for their teaching. Take them step-by-step through the document and 
workshops on how it would look in the classroom. 

One of the most important things the Teacher Associates learned was how to work with 
the science committee in the school, get them to meet effectively on a regular basis and 
continue to pursue changing science education in that building. That was an important 
element to making change in the building, and drew more people in than we ever had 
before. Now we had, particularly in our partnership schools, active science and math 
committees looking at their curriculum. So a lot of curriculum change started to happen, 
to incorporate the new science and math materials that were available. So that strategy 
pulled in a lot more people who would participate on a continuing basis than what we 
had with the summer institutes. At the institutes, people would get excited, but if there 
was no leadership back in the school, it would be gone in six or seven weeks after the 
summer was over. 

VISMT began with four full-time Teacher Associates in 1995, then expanded the cohort to 
twelve full-time positions. While the original intent was that a Teacher Associate would be 
released for a year, many worked with VISMT for two years before returning to their classrooms 
to provide local leadership. The Teacher Associate model is still in use in VISMT, although it 
has been scaled back somewhat. The current Executive Director explained: 

In Phase II we still bring teachers in, usually for two years. We found it took the first 
half of a year to get the skill set they needed, so one year was not enough. We have 
multiple options now. Some “pure” Teacher Associates, some hybrids of where they 
work and how they spend their time. Part of the reason for hybrid forms was economic, 
based on less NS F funds in Phase II. We looked for more cost effective ways to build 
teacher leadership capacity, but not the same way for all people. Also, in the early days, 
we were able to go out an almost hand-pick the best candidates for the positions. What 
we found over time was that there were good people interested in this kind of work, but 
not all of them are in a position to take a year to come work full time in the Teacher 
Associate model. We started looking at how to build the model around the people, 
keeping the parameters that were important to us but giving some flexibility in what the 
model would look like. And it was also partly driven by the recognition that there is 
different kind of work that we needed teachers to do. Some of it was done well in the 
traditional Teacher Associate model, and some of it was done well in other ways. The 
other thing is that as we’ve developed a cadre of former Teacher Associates, we now 
have some eyes and ears and talent in the field that we didn ’t have then, so there’s less 
need for bringing in a full cohort of people. We now have about 80 people. In Vermont 
you can make a lot of impact with 80 leaders. 
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Moving from the centralized summer institutes to a regional format built around shorter, targeted 
conferences and on-site assistance from Teacher Associates was a key to VISMT’s impact on 
schools. Furthermore, the shift from working generally on inquiry and standards-based teaching 
to a specific effort to translate the Framework into local curriculum provided the “hook” for 
sustained VISMT presence in its participating schools. The next step was a focus on data and 
improvement planning. This is discussed in the section below. 

Institutes have not disappeared from the Vermont landscape. The Vermont Math Institute and 
the Vermont Science Institute are three-year programs operated by institutions of higher 
education, providing elementary teachers with sustained experiences in implementing exemplary 
curriculum materials and using standards-based instructional strategies. VISMT is a partner in 
these institutes, but is not taking the lead. A staff person commented that these institutes were a 
way to keep an active higher ed connection. 

School Partnerships 

Like professional development, VISMT’s approach to school partnerships underwent 
considerable evolution and became better integrated into the initiative’s overall plan. Originally, 
the partnership program was to catalyze innovative ideas and build local school/community 
collaboration. Results were mixed, and VISMT leaders realized a different approach was 
needed. A PI and a former Executive Director described the situation: 

From the first, we had the granting program and the summer institutes separate, but the 
intention was to build long-term relationships with school-based teams. The incentive 
grants were a mechanism, using the challenge grant model. Successful recipients of 
those grants became partnership schools who would make a long-term commitment to us. 
We saw the grants as starter-fires for loccd capacity. We didn 7 see the schools becoming 
“model schools. ” Eventually we changed the concept of partnership to include cdl 
schools in the state. We would find a way to partner with everyone. 

By about the second year, / was pretty discouraged. I watched us go through a grant 
program, funding grants to lots of schools. 1 got excited about some of the proposals, but 
when you would go out nothing had recdly started yet, or they ’d lost the person who 
wrote the proposal and didn 7 know what to do. We had a very rigorous summer 
institute, very intense. We had follow-up planned for cdl participants. At the first follow- 
up, if 60 percent showed up we were happy. At the second follow-up we were down to 20 
percent. At the last one you could sit around a table together. We were giving cdl this 
money away and nothing was happening. We did a lot of soul searching about what to 
do. So we came up with this thing called partnership schools where we attempted to get 
some strong guarantees that things would happen. . . We had the idea that if we sat 
clown with people from the school and draw up a contract. Treat VISMT as a partner — 
what we can give them, what they can give us. We asked them to file quarterly reports 
around documents that indicated change. We sat down with teams in the schools to help 
them do that, so our presence in the schools began to increase. 

The shift to “partnership schools” in 1994 represented a more formal, continuous relationship 
between VISMT and participating schools. Partnership schools received focused assistance to 
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examine their curricula, to build relationships with local resources, and to develop a more 
systemic outlook to improving their instructional programs. When the Teacher Associates came 
online, a large part of their time was spent in partnership schools. VISMT’ s role changed from 
simply providing funds to connecting schools to people and organizations with expertise to help 
them in their reform efforts. In return, the SSI expected a greater degree of documentation and 
accountability from the schools it worked with. 

As the new Frameworks were rolled out and participation in the state assessment system was 
mandated, VISMT expanded its work beyond the original partnership schools, which took a 
comprehensive approach to reform, to try to reach all schools in some way. With a growing 
cadre of Teacher Associates and other leaders arising from the body of summer institute 
participants, VISMT tried to find an appropriate “hook” for developing a relationship with each 
school. Initially, the expanded effort had mixed results. Not all schools were ready, or saw a 
need, for the kind of team-oriented planning and improvement process that VISMT was 
promoting. That situation changed with the passage of new legislation in 1997. The Executive 
Director described it as follows: 

One of the events early in Phase II was the passage of Act 60 in 1997, the Equal 
Educational Opportunity Act. It was a fiscal equity act, looking at the funding formula 
for schools, and a school quality act. Act 60 set up expectations for schools that were 
different from what had been. One was that schools would start using data in more 
serious ways, that schools would examine their student performance data in context of 
other data and create an annual action plan to focus professional development, resource 
allocation, energy, etc. That’s made a huge difference. What we’ve had to do as an 
organization is to become responsive to the action plans. If schools have decided where 
they’re going to put their emphasis and spend their time in professional development, we 
need to be part of helping them get what they need. So we spend a lot more time helping 
people do data analysis, translating the data into action plans, and then implementing 
those plans. That has fundamentally changed the way we work. In the early days it was 
much more likely for VISMT to develop a week-long institute on inquiry and have people 
come. Now it’s more likely that we would have a group of schools, all of which identified 
inquiry as part of their action plan. It would come from them. So now it’s more of a 
response/ dissemination model than a creation and see who comes kind of model. 

Act 60 mandated a team-oriented action planning process at each school, driven by analysis of 
student performance data. Many schools, however, were not prepared to do this in a meaningful 
way. With sanction from the Department of Education, VISMT capitalized on the grassroots 
needs and concerns, using Act 60 as a context for working with schools on their mathematics and 
science programs. VISMT developed and conducted regional conferences on data analysis. 
Teacher Associates worked with schools to examine and analyze their data, identify program 
needs, and develop action plans. In so doing, they emphasized the importance of aligning local 
curriculum and instruction to the state standards and assessment. The push for alignment had 
taken on new significance for local staffs. And, as noted in the quote above, VISMT now 
identified focal areas for professional development and technical assistance based on patterns 
arising from the school action plans. 
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In addition to the data analysis conferences, VISMT addressed the need to enhance local 
capacity to engage in data-informed improvement planning by providing leadership academies 
for teams of local administrators and teachers. These academies emphasized involving the entire 
school community in the process of improvement. They also highlighted the important roles for 
administrators in supporting implementation of high-quality mathematics and science programs. 
Schools with teams participating in the leadership academies were better positioned to build on 
the on-site assistance provided by the Teacher Associates. 

The merging of VISMT’ s professional development and school partnership initiatives into an 
integrated support system was a key development. The project also benefited from changes in 
the state policy environment, showing an opportunistic ability to use these changes to increase 
the breadth and depth of its work in schools. 

Equity 

Equity began in VISMT with the feel of an add-on, with an emphasis on gender equity issues and 
Family Math/Science nights. Development of Equity Benchmarks in 1993 showed an early 
commitment to producing tangible outputs from the initiative. Later, as work with partnership 
schools matured, the SSI’s approach to equity expanded to focus more on poverty-related issues. 
Low-achieving, high-poverty schools received priority for attention in VISMT’ s regional 
conferences and site -based technical assistance. Some sites were assigned Equity Associates 
(Teacher Associates with specific training in equity issues) as part of a pilot program to infuse 
equity considerations into data collection and improvement planning. Data analysis work with 
schools emphasized disaggregation of data to look for student subgroups who were lagging 
behind. Work on local curricula examined opportunities for all students to have learning 
opportunities that addressed all the standards. A partnership with the University of Vermont 
introduced the “Complex Instruction” model as a means of promoting equitable learning 
environments in elementary classrooms. A Parent and Community Guide, “The Opportunity to 
Learn” was widely disseminated and formed the basis for broadening equity-related discussions 
beyond educators. The equity strand was a highly-visible and important component of VISMT’s 
later activities, and equity was one of the cornerstones of its work with schools and districts. SSI 
leaders wanted it to be woven throughout the work, not just as a separate initiative. 

Teacher Preparation 

Impact on higher education, while acknowledged as important, was not a major part of the 
VISMT effort. The primary strategy in Phase I, awarding grants for course development, was an 
attempt to “sow the seeds” of change by stimulating innovation. Like the early strategy for 
school partnerships, results of the grant program were mixed. The effort does not appear to have 
had a widespread, lasting effect on teacher preparation programs. A Board member 
acknowledged this, commenting that, “in terms of actual teacher education, I don’t think (the 
impact) was very big. We weren’t running courses or doing major changes in curriculum. There 
weren’t many resources.” 

Unlike the partnership schools initiative, however, VISMT did not find the right “hook” to work 
with the institutions in substantive ways. An effort to bring about substantial changes to the 
credentialing regulations showed early promise, but has not yet resulted in the policy changes 
envisioned. 
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In Phase II, VISMT was a partner with a prominent mathematics faculty member in offering the 
Vermont Mathematics Institute, a three-year program for elementary teachers leading to a 
masters degree with an emphasis in mathematics education. The positive reaction from 
participants led to creation of a parallel Vermont Science Institute for elementary teachers at 
another institution. While these initiatives have been well-received and have been successful as 
graduate-level teacher enhancement, VISMT’ s ability to affect teacher preparation at the 
programmatic level has been minimal. 

Building Support, Addressing Concerns, and Making the Case 

VISMT’ s structure as a collaborative effort and its status as an independent, nonprofit 
organization were a direct response to the inclusive, participatory culture existing in Vermont. 
Both by desire and by necessity, project leaders had to be sensitive to who was involved in 
planning and implementation. Overall, project personnel felt they had been successful at 
including everyone who wanted to be a part, as reflected in a Pi’s comment: 

I think the level of trust and working partnership among the stakeholders is what makes 
this thing work. We don ’t have to go out and beg for a place at the table. The attribution 
then becomes almost moot. VISMT couldn ’t have done any of it by itself but it was at the 
table when it happened 

Project staff noted that some of the early critics, from higher education, had come around to lead 
portions of VISMT’ s current slate of activities: 

One person comes to mind who doesn ’tfeel he was included as much as he wanted. He 
was a mathematician. Interestingly enough, he has come forth to develop a new initiative 
with the Department of Education and VISMT called the Vermont Math Institutes. But 
for the first four or five years of VISMT he was negative toward the project, felt he wasn 7 
included in the planning process. I think what happened was that he was included but 
didn 7 end up as a co-PI or on the Board. 

There was one period of time [in the later part of Phase I] that was a very difficult time 
because the Burlington Free Press wanted to expose the fact that we had spent $7 million 
but didn 7 have much to show for it. Right after that article appeared, we had a meeting 
of state college people to offer them grants to develop science and math courses for 
undergraduates. Someone stood up at that meeting and said we should send the money 
back to Washington. Ironically, the person who said that is now the person who is 
developing the Vermont Science Institutes that is a mirror of the Math Institutes. So you 
never know, two years later she ’s back wanting to play. 

Despite these successes at drawing-in faculty members who were initially critical, building 
bridges to higher education remains one of VISMT’ s greatest challenges. A former Executive 
Director recalled: 

We had terrible problems in the beginning getting anything going at the university. It 
wasn 7 the fact that we didn 7 try to get people involved, but we just didn 7 push the right 
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buttons. I had come from higher ed and it seems like I had gone off into Never-Never 
Land when / joined VISMT. My colleagues thought that perhaps I was going in the 
wrong direction professionally. . . [Involvement of] higher ed is still not where I would 
like to see it. 

Concerns about VISMT arose from a lack of visibility during the early period of Phase I, when 
most of the work was at the policy level and most of the contact with local educators was 
through the summer institutes. The local-control tradition in Vermont led to some suspicion of 
centralized efforts that appeared to create more bureaucracy, and VISMT received some negative 
attention from concerned citizens (such as the Burlington Free Press “investigation” mentioned 
in the quote above). VISMT’ s response was to be an open organization, providing information 
and inviting participation and discussion. As the work of the SSI moved out into the local level, 
with the advent of Teacher Associates and partnership schools and the focus on promoting and 
implementing the Frameworks, VISMT’ s public image was enhanced. This led to emergence of 
another issue, described by the Executive Director at the time: 

Not early on, but after 1995, we became known as the group who could deliver in the 
state. The State Education Department began to get a reputation as being useless. / 
could not have that happen, because they were an important partner. 1 had to work very 
hard with superintendents and others in the state that the Department was a vital partner 
in our work, and what we were doing was in cooperation with the Department. The 
Commissioner at that time and I began appearing a lot together at conferences to talk 
about the work we were doing and supporting each other. Other people at the 
Department were saying, we’re never going to get any federal money because you guys 
are getting it all. And it was true to an extent. To inject $9 million in a state of this size 
is significant, and overshadows anything else you might get. 

Both at the state and local levels, VISMT came to be seen as the major player in mathematics 
and science education, and its leaders became the recognized “voice” for mathematics/science 
reform. 

NSF also had concerns about the lack of visible results in the early years of VISMT, for many of 
the same reasons as the local concerns discussed above. The SSI’s policy work was critical to its 
plan, but did not lend itself to attributions of school-level impact. Information and assurances 
from VISMT leaders and state policy leaders, together with the introduction of the Teacher 
Associates and school partnership programs, addressed the NSF concerns about the project’s 
strategy. 

After an early, unsuccessful relationship with its first external evaluator, the project began to be 
more organized and intentional in its efforts to tell its story. The later Executive Directors were 
specific in their focus on using quality information to monitor implementation, make sound 
decisions, and keep stakeholders informed. Using both internal mechanisms and a new external 
evaluator, Western Michigan University, VISMT collected an array of data about its operations 
and its outcomes. The lack of a mandated state assessment until 1997 was a limitation in 
gauging student impact, but other information collected and reported highlighted the project’s 
influence at both the state and local levels. In its reports, VISMT tried to illustrate the tangible 
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and growing results of the state reform, with enough attribution to satisfy NSF but still 
emphasizing the collective nature of the statewide endeavor. VISMT’s relationship with its 
partners and funders was (and continues to be) an ongoing balancing act, gamering enough credit 
and credibility to enable the work to continue but not standing out too far from the collection of 
players who all contribute to the result. Overall, VISMT leaders have been proactive in building 
support, and have reacted strategically when problems arose. 

Looking Ahead 

As VISMT nears the end of its Phase II funding, the question of its continued viability is a 
natural one. This is an issue that the staff and Board have addressed since the later stages of 
Phase I. As a nonprofit organization, VISMT has always been dependent on external sources of 
funding to operate. Initially, the great majority of its funds were from the SSI award. VISMT 
has continually sought to expand and diversify its income streams, beginning with operating the 
state’s Technology Literacy program as a companion to the mathematics/science reform. The 
organization now has a variety of programs, including literacy and a technology-based 
Interactive Learning Network, and is beginning to establish a fee-for-service model for providing 
even greater assistance to schools in their data analysis/action planning process. Organization 
leaders are consciously positioning VISMT to capitalize on opportunities within the current 
reform environment. In so doing, however, concerns have arisen among a small number of 
leaders that VISMT is reducing its attention and commitment to mathematics and science before 
the work is completed. Certainly the portfolio of VISMT initiatives has expanded beyond an 
exclusive focus on mathematics, science, and technology. Perspectives differed among persons 
interviewed whether this represents a move away from its original mission or a broadening of 
that mission to address quality education in general. These different perspectives are evident in 
the following comments, the first from a current staff member, the second from a Board member: 

We are thinking more entrepreneurial ly now. Our partnerships are expanding; the 
variety of work is more wide open than ever before. Ten years ago, if it wasn 7 science or 
math we probably weren 7 doing it. Now we see ourselves as school change, systems 
change, policy work, literacy, teacher education, with science and math still on the 
agenda. The agenda and horizon have expanded. Looking out around us and asking 
what else we can be doing. 

In the last three years staffing has gone from 12 people to 60, with at least five non- 
science/ math related projects housed at VISMT. What it looks like is that VISMT will 
become a collection of lots of entrepreneurial projects. What concerns some of us who 
were around from the beginning is that we’ve really not completed the job in science and 
mathematics yet, and we see that as being underplayed in what’s going on now. There 
has been conflict on the Board on this issue, how all the different projects tie together. 

It’s safe to say that the organization is influx right now. 

In order to manage these differences of opinion over the direction the initiative should take, the 
business plan that has guided VISMT’s transition process from federal funding and the NSF 
cooperative agreement to a fully independent organization with a sustained presence in the state 
has been driven by a Board Futures Committee. The committee gave unanimous conceptual 
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approval twice, and voted unanimously, with one abstention, for the final plan that broadened the 
work of the initiative as described. 



Impact and Residue of the SSI 

Policy 

Because VISMT was (and continues to be) closely connected to the Department of Education, it 
has exerted substantial influence on the policies that shape Vermont’s education system: 

• The SSI was the major mathematics/science player in development of the 
Frameworks. While the final document did not have the integrated structure that 
VISMT leaders originally envisioned, it was strongly linked to the national standards 
and set forth an ambitious vision of what mathematics and science teaching and 
learning should be. The development took much longer than expected, but once they 
were available the Frameworks provided the platform from which VISMT launched 
its meaningful work with schools. 

• The SSI had considerable influence on the development and implementation of 
Vermont’s state assessment system in mathematics and science. VISMT played the 
lead role in expanding the portfolio assessment and identifying other assessment 
tools, such as the New Standards Reference Exam, that were consistent with the 
outcomes envisioned in the state standards. When the legislature mandated local 
participation in the state assessments and began to attach accountability measures to 
performance, Vermont educators could have confidence that the assessments would 
yield meaningful information. 

• Once data began to be available through the assessment system, VISMT leaders were 
at the table in framing portions of the legislation and policies addressing school 
quality and improvement planning. Fike the Frameworks, the school quality 
provisions of Act 60 provided important leverage and context to enable VISMT to 
work with schools in a sustained manner. 

• Prior to VISMT, there was no real professional development “system” in Vermont, 
only a fragmented collection of regional initiatives. Moreover, local professional 
development was inconsistent at best, nonexistent at worst. Through its Summer 
Institutes and the subsequent RPDIs, VISMT sent a message that access to 
professional development was important for all, and that consistency across the state 
was a critical element. The Department of Education uses VISMT’ s regional 
framework to organize some of its other professional development efforts. In 
addition, VISMT has contributed to a shift in perspective of what constitutes 
professional development. The work of the Teacher Associates in providing ongoing, 
site -based assistance to school teams has broadened the image of professional 
development beyond just workshops and courses. The work of school teams to align 
local curricula, to support each other implementing new teaching practices, and to 
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analyze available data to identify program needs is increasingly acknowledged as 
valuable professional growth. 

VISMT is seen as the “voice” of mathematics and science reform in the state, and has an 
expected place at the table in discussing, planning, and implementing policy at the state level. 

Local Impact 

The changes in strategy that VISMT instituted resulted in much broader and more meaningful 
local impact than would likely have been the case with its original design. These impacts 
include: 

• The original Summer Institutes developed a cadre of local advocates and leaders for 
quality mathematics and science education, but local opportunities and support for their 
leadership were often lacking. The subsequent work with school teams through the 
Partnership Schools gave these leaders a context through which to work. Local 
leadership capacity was further enhanced through the Teacher Associates initiative, 
which to date has involved close to 80 teachers in sustained leadership work. 

• VISMT has worked with over two-thirds of the schools through its partnership 
program. This work has resulted in local curricula better aligned with the Frameworks, 
and more skillful use of data to identify curricular and professional development needs 
and develop improvement plans. State leaders noted that the quality of the mandated 
action plans tends to be high in schools that have strong ties to VISMT. 

• The use of quality instructional materials in classrooms has increased, particularly since 
the regional Science Education Cooperatives began making materials available on a 
circulating basis. 

• Evidence of increased use of standards-based instructional strategies continues to 
accumulate. VISMT leaders acknowledge that sustained change at the classroom level 
is a slow process. In particular, the mathematics portfolio appears to have influenced 
practice at the elementary and middle grades. Science lags somewhat behind, in part 
because of the earlier emphasis on mathematics in the assessment rollout. 

• Data on student performance in mathematics and science only began to be available 
with the mandate for statewide participation in the assessment system, so comparisons 
to the early years of the SSI are difficult. Current data indicate that schools involved 
with substantial implementation (aligned curricula, level of professional development 
participation, ongoing data-driven discussions of program improvements) are also 
demonstrating measurable improvements in student learning. 
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Factors That Shaped the Vermont SSI Story 



Examination of the context, players, and events that constitute VISMT’s story to-date indicate 
several factors that were key to the SSI’s successes and challenges. 

> Timing is Everything: VISMT rode a developing reform wave, rather than trying to 
generate the wave itself 

At the time the SSI was funded, Vermont was early into its drive toward a standards-based 
system. The Commissioner was leading the state (some would say dragging the state) toward a 
standards-driven orientation. Given his push for standards in general, VISMT did not have to 
justify the importance of standards; its role was to supply the mathematics/science expertise to 
the standards development. Similarly, one of the needs noted in the SSI proposal was the lack of 
an overall state system to lend focus and direction to fragmented local efforts. The 
Commissioner’s agenda was to strengthen and bring coherence to the state system; VISMT rode 
the coattails of that agenda. 

VISMT leaders were opportunistic, recognizing opportunities in the state reform landscape and 
capitalizing on them to provide context for working on the mathematics and science reform. The 
project’s work in standards, assessments, data analysis, and school improvement planning all 
arose from seeing the next crest in the reform wave and getting in position to ride it. 

> VISMT worked as part of, but also apart from, the official system 

VISMT was not the Department of Education, but was closely connected to it. Thus, the SSI 
was in a position to have a strong influence on the development of state policy, while at the same 
time not being an agent of the state in working with local schools or its other partners. VISMT 
provided greater science, mathematics, and technology expertise and leadership than what the 
Department of Education could provide itself, and so was seen (for the most part) by the 
Department as a welcome partner. It did not threaten existing power bases. 

In the higher education sector, on the other hand, VISMT’s “outside” status produced more 
challenges. Higher education faculty were the mathematics and science leaders of the state (even 
though they rarely worked on a statewide basis), and some friction occurred when VISMT 
became the acknowledged leader of the reform effort. By recruiting some of its key staff from 
higher education and including others as Board members, VISMT sought to tap into the existing 
leadership and to position itself to influence higher education. For the most part, VISMT’s 
inclusive approach won over its critics and VISMT used its higher education connections 
effectively to support the K-12 work. But it has not been able to leverage reform in higher 
education itself to the degree it expected. 

> VISMT tended to policy and infrastructure first, and used that as the leverage for its 
local work 

The fierce local control culture of Vermont contributed to the fragmented state that existed at the 
beginning of the SSI. VISMT’s work on state standards and assessments formed a unifying 
vision and framework to guide work (and expectations) at the local level. Similarly, the 
statewide development of leadership, through the institutes and Teacher Associates, built a 
network of knowledgeable people who shared the basic vision and could advocate for it in the 
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districts. In rolling out the Frameworks, in providing training on scoring portfolios, in working 
with schools on data analysis for Act 60, VISMT could take the position of being available to 
help schools respond to state expectations. Rather than pushing standards-based mathematics 
and science as the “right thing to do”, VISMT took a pragmatic approach — “let us help you do 
what you’ll have to do anyway.” 

> Vermont was the right scale for the size of the SSI investment and strategy 

VISMT initially thought it could work successfully on a statewide basis using a centralized 
approach. This approach was problematic. But with fewer than 400 schools in 61 supervisory 
unions, the size of the Vermont system matched VISMT’s eventual strategy of working at the 
school level to support implementation of the Frameworks, assessments, and other state-level 
changes. While VISMT did not establish the close relationships it desired with all the schools, it 
managed to have significant contact with the great majority. The state’s size and scale also 
enabled the regional support system to have a manageable number of regions, each responsible 
for a reasonable number of schools. This enhanced the effectiveness of the Teacher Associates, 
regional institutes, and Science Education Cooperatives. Since the state was relatively resource- 
poor, the SSI funds were a major stimulus to the Vermont reform. The level of NSF support was 
sufficient for the scope of activities at the scale required in the state. Furthermore, the NSF 
funds enabled structures and programs to be established, which were then augmented with other 
funds from state, federal, and private sources to maintain and expand the activities. 

> For the most part, VISMT had the right people in the right places and kept them long 
enough to get things done 

Turnover in leadership is a major obstacle to long-term, meaningful change. In this area, VISMT 
was fortunate in several respects. Turnover in key leadership was very low. For example, the 
Executive Director noted that the same Governor had been in place throughout VISMT’s 
existence, and they could count on him for consistent support. Membership on the VISMT 
Board was “remarkably stable for a nonprofit Board,” lending consistency to the Board’s 
oversight of the initiative. When turnover did occur, as in the Commissioner’s departure, his 
replacement was already closely connected to VISMT and provided continuity in vision and 
direction from the Commissioner’s office. Other key state leaders also came from the VISMT 
ranks. These connections contributed to VISMT being seen as the major player for mathematics 
and science. And, of course, the VISMT Executive Director position was critical. After a rocky 
beginning, the Executive Director became VISMT’s driving force. The two persons holding that 
position (after the original) provided the strategic vision required for the SSI to adapt its work to 
take advantage of opportunities arising in the state reform context. 

> VISMT leaders learned lessons and applied them to reshape the initiative 

As mentioned above, VISMT’s two latest Executive Directors (and other staff as well) were 
highly adaptive, and adept at learning from experience to make VISMT stronger and more 
effective. Their attention to data as a formative tool set a strong example for refining/shifting 
strategies based on clear vision and good information. Examples of shifts in VISMT’s approach 
that resulted from lessons learned include: focusing on schools as the unit of action and change, 
rather than trying to work with individuals in institutes and sending them back as hopeful 
catalysts; creating support systems to work on-site and embed the reform work in the operation 
of the school; creating meaningful partner relationships, not just handing out money and 
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expecting results; and providing specific, targeted outreach to invite participation and provide 
resources to schools serving populations of higher-poverty students. 
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